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ABSTRACT 


"The  Marine  Corps  Combat  Readiness  Evaluation  System 
(MCCRES)  was  designed  to  provide  timely  and  accurate  infor¬ 
mation  concerning  the  ability  of  active  and  reserve  forces 
to  perform  assigned  combat  missions.  To  provide  this  infor¬ 
mation,  units  are  subjected  to  simulated  combat  problems. 
Their  performance  is  observed,  evaluated,  and  reported  by 
evaluators  from  within  the  Marine  Corps.  These  evaluators 
are  key  to  the  collection  of  valid  evaluation  data.  If  the 
evaluator  is  not  effective,  then  the  MCCRES,  as  an  evalua¬ 
tion  system,  is  ultimately  ineffective  in  determining  a 
unit's  "combat  readiness.^ 

The  purpose  of  this  thesis  is  to  analyze  the  selection 
and  use  of  evaluators  in  the  MCCRES.  The  current  structure 
ar.d  process  used  for  management  control  in  the  selection  and 
subsequent  education  of  MCCRES  evaluators  was  investigated. 
MCCRES  evaluators  were  interviewed  and  their  recommendations 
for  improving  evaluator  effectiveness  were  compared  with  the 
existing  MCCRES  models.  The  comparison  resulted  in  a  set  of 
recommendations  to  modify  the  current  models. 
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INTRODUCTION 


A .  GENERAL 

In  1978  the  Marine  Corps  initiated  a  new  evaluation 
system  to  be  used  throughout  the  Corps.  This  system,  the 
Marine  Corps  Combat  Readiness  Evaluation  System  (MCCRES) , 
was  developed  to  provide  a  means  to  effectively  measure  and 
then  report  findings  as  to  whether  a  given  unit  was,  or  was 
not,  prepared  to  perform  its  assigned  combat  mission.  This 
new  system  was  to  provide  both  standardized  measurement 
objectives  and  reports  for  the  combat  unit  evaluated,  regard' 
less  of  the  uniqueness  of  its  assigned  combat  mission. 

MCCRES  can  be  viewed  as  part  of  the  Marine  Corps'  management 
control  system,  in  that  it  is  a  means  by  which  the  Marine 
Corps  assures  its  resources  are  used  to  meet  its  objectives 
[Anthony  and  Herzlinger,  1980] . 

After  five  years  of  operating  this  evaluation  system, 
it  is  perhaps  an  appropriate  time  to  review  the  effective¬ 
ness  of  the  various  inputs  into  the  MCCRES  in  meeting  the 
originally  designed  objective:  effective  measurement  of 
combat  readiness.  One  important  input  into  MCCRES  is  the 
evaluators.  Their  effectiveness  in  the  proper  marking  of  a 
given  observation  contributes  directly  to  the  ability  of  the 
system  to  meet  its  designed  operational  objectives.  The 


selection  and  training  of  these  individuals  has  the  potentia 
to  affect  the  quality  of  the  MCCRES. 


B.  OBJECTIVE 

The  objective  of  this  thesis  is  to  analyze  the  selection 
and  use  of  evaluators  in  the  MCCRES.  In  order  to  achieve 
this  objective  an  answer  to  the  following  quei  on  is  sought 
"Can  a  model  be  developed  to  assist  in  the  ef  ient  selec¬ 
tion  and  education  of  ootential  MCCRES  evaluc  so  that 


the  evaluators  will  perform  at  a  predetermined  level  of 
effectiveness? " 

If  a  model  can  be  developed,  a  secondary  question  then 
must  be  asked:  "What  attributes  of  the  management  control 
system  (i.e.,  its  structure  and  process)  will  this  model 
provide?"  The  attributes  provide  the  concerned  commander 
with  a  technique,  or  tool,  that  can  be  used  for  more  effec¬ 
tive  and  efficient  management  of  limited  resources. 

C.  SCOPE  AND  METHODOLOGY 

The  scope  of  this  thesis  is  limited  to  the  function  of 
the  evaluator  within  the  MCCRES.  Of  primary  concern  is  how 
the  evaluator,  as  a  resource,  can  be  more  effectively  used. 
This  can  be  accomplished  by  modifying,  directing  and  chan¬ 
neling  the  influence  he  has  on  MCCRES  as  a  result  of: 

(1)  qualifications  and  attributes  he  brings  with  him  (e.g., 
his  past  experience) ,  (2)  attributes  acquired  immediately 


oefore  a  MCCRES  (e.g.,  attendance  at  a  MCGEES  evaluu 
school)  ,  and  (3)  those  attributes  acquired  durinc  th. 
of  a  MCCRES  (e.g.,  individuals  that  can  influence  h: 
ior  during  the  conduct  of  a  MCCRES) . 

The  method  used  to  investigate  the  area  of  cor.ce 
to:  (1)  conduct  a  field  study  to  gather  information 

contributed  to  evaluator  effectiveness,  (2)  compare 
information  with  that  of  accepted  manacement  control 
and  (3)  take  the  results  of  the  comparison  and  dovol 
model  that  would  help  the  commander  to  execute  more 
control  over  the  operation  of  the  evaluation  system. 

The  model  provides  the  commander  a  means  to  impr 
effectiveness  of  his  evaluators  in  tne  measuring  of 
vation  and  then  reporting  that  information  into  the 
However,  the  model  is  only  a  visual  means  of  describ 
a  certain  "structure"  and  "process"  allows  for  a  bet 
of  conducting  business.  It  is  paramount  to  remember 
is  the  knowledgeable  commander  and  the  use  of  his  no 
ment,  applied  to  a  particular  problem,  that  produces 
resulting  decision  that  actuates  the  efficient  and  e 
operations . 
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II .  BACKGROUND 


A.  GENERAL 

This  chapter  addresses  the  purpose,  scope,  structure, 
and  process  of  the  Marine  Corps  Combat  Readiness  Evaluation 
System  (MCCRES) .  The  focus  will  be  on  MCCRES  accomplishing 
a  required  need  as  seen  from  the  Headquarters,  Marine  Corps 
level . 

Prior  to  1978  the  Marine  Corps  did  not  have  a  uniform 
method  for  the  measurement  of  accepted  standards  in  deter¬ 
mining  a  given  unit's  combat  readiness.  Unit  operational 
readiness  tests,  or  "Tac  Tests"  were  conducted  with  stan¬ 
dards  individually  constructed  and  implemented  at  essen¬ 
tially  every  major  wing  and  ground  tactical  organization 
throughout  the  Corps.  Although  the  "Tac  Tests"  may  have 
been  useful  within  the  units  that  developed  them,  there  was 
not  any  organized  way  to  compare  results  between  similar 
units  of  other  organizations.  It  was  this  lack  of  uni¬ 
formity  in  determining  combat  readiness  that  prompted  the 
Commandant  of  the  Marine  Corps  to  establish  an  evaluation 
system  that  would  provide  a  single  standard,  a  method  of 
application,  and  procedures  for  measurement  of  combat  readi¬ 
ness.  This  was  done  on  1  July  1978  after  considerable  con¬ 
sternation  and  thoughtful  planning.  [Erickson,  1981] 
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B.  PURPOSE  OF  MCCRES 

One  of  the  most  crucial  problems  facing  Marine  Corps 
commanders  is  the  maintenance  of  an  "effective"  military 
capability.  Creating  comoat-ready  units  in  time  of  war; 
establishing  standards  and  priorities  for  training,  procur¬ 
ing,  and  staffing  during  periods  of  relative  peace;  and 
assuring  a  Marine  Corps  capability  that  deters  potential 
adversaries  from  dangerous  adventures  have  long  been  the 
central  mission  of  the  U.S.  Marine  Corps.  [DARPA,  1977] 
Measuring  "military  effectiveness"  can  be  accomplished 
through  a  formal  system  of  measurement  that  maintains  a 
single  standard  and  is  uniformly  applied  throughout  the 
entire  Corps.  [DARPA,  1979]  Within  this  paper  no  distinc¬ 
tion  is  made  between  the  terms  "effectiveness"  and  "readi¬ 
ness."  For  purposes  of  clarity,  the  word  readiness  will  be 
used  throughout  this  paper.  [DARPA,  1977] 

A  major  difficulty  in  measuring  the  readiness  of  a  unit 
results  from  the  fact  that  the  unit  is  not  evaluated  under 
real  combat  conditions.  Instead,  it  is  evaluated  while 
executing  several  exercises  representing  typical  operations 
which  the  unit  is  supposed  to  be  able  to  execute  while 
accomplishing  its  mission  under  combat  conditions.  To  avoid 
the  difficulty  of  achieving  perfect  duplication  of  combat 
conditions,  it  is  common  to  replace  the  question  "Can  the 
unit  do  the  job?"  by  "How  'close'  is  the  execution  to  the 
doctrine."  It  is  assumed  that  units  which  follow  the 


doctrine  closely  are  likely  to  be  able  to  do  their  job:  mis¬ 
sion  accomplishment  under  conditions  of  conflict.  [Barzily, 
1980] 

The  purpose  of  MCCRES  is  to  use  simulated  combat  to 
evaluate  the  readiness  of  Marine  units  in  their  "effective" 
accomplishment  of  assigned  missions. 

C.  SCOPE  OF  MCCRES 

Upon  implementation  of  the  MCCRES,  specific  tests  were 
designed  for  use  in  the  evaluation  of  all  units  in  the  Corps 
that  perform  a  combat  associated  mission. 

MCCRES  was  adopted  in  July,  1978,  to  provide  standard¬ 
ized  evaluation  policies  and  procedures,  and  to  provide  the 
definition  of  standards  for  mission  performance  that  are 
applicable  to  evaluation  of  the  combat  readiness  of  Fleet 
Marine  Force  units.  Specifically,  MCCRES  provides: 

- Performance  standards  (MPS)  based  on  assigned 

missions . 

- A  standardized  evaluation  process. 

- A  standardized  reporting  system. 

- Feedback  to  units  indicating,  strengths  and 

weaknesses . 

The  entire  MCCRES  system  is  contained  in  a  ten  volume 
order.  Volume  I  outlines  the  evaluation  system  and  defines 
the  Mission  Performance  Standards  (MPS) .  The  application  of 
these  MPS's  is  the  backbone  of  the  evaluation  process.  It 
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can  be  argued  that  MPS's  are  the  system  designers'  approach 
to  devising  doctrinally  correct,  standardized  criteria  for 
evaluating  a  unit  tactically.  In  developing  this  criteria 
the  designers  tried  to  eliminate  subjective  evaluation  and 
to  promote  quantitative  analysis.  [Rothwell,  1979] 

D.  ELEMENTS  OF  MCCRES 

MCCRES,  as  a  formal  evaluation  system,  must  possess  two 
elements:  (1)  structure,  and  (2)  process.  The  structure  is 

the  organizational  arrangements  and  information  constructs 
that  facilitate  the  process.  The  process  is  the  set  of 
actions  that  take  place.  Stated  more  simply,  structure  is 
what  it  is  and  process  is  what  it  does.  In  studying  the 
human  body,  for  example,  one  needs  to  understand  both  its 
anatomy  and  its  physiology.  [Anthony  and  Herzlinger,  1980; 
Anthony  and  Herzlinger,  1975]  These  two  elements  are  used 
throughout  this  thesis  to  focus  on  how  MCCRES  functions  as 
a  system. 

1 .  MCCRES  Structure 

The  structure  of  MCCRES,  as  an  evaluation  system 
provides  the  organization  for  control  and  assignment  of 
responsibilities  in  the  collection  and  evaluation  of 
appropriate  data  to  be  used  in  the  determination  of  unit 
readiness.  [MCO  3501.2,  1977] 

It  is  the  structure  of  MPS's  that  deal  with  specific 
operational  functions  and  missions  that  a  unit  might  be 
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expected  tc  accomplish  in  combat.  Mission  oriented  MPS ' s 
for  an  infantry  battalion  might  include  attack,  defense, 
tank-infantry  operations,  mechanized  operations,  and  surface 
assault. 

The  structure  of  MPS's  can  be  further  broken  down  to 

that  of: 

- Task  to  be  performed. 

- Requirements  which  must  be  accomplished  to  fulfill 

the  task. 

- Conditions  under  which  the  task  is  tc  be 

performed . 

The  relationship  of  MPS's  elements  are  reflected  in  Figure 

2.1. 


Figure  2.1  Relationship  of  elements  to  MPS,  MCCRES 

[MCO  3501.2,  1977] 

It  is  the  breaking  down  of  MPS's  into  smaller  more 
manageable  parts  (observations)  that  allow  the  MCCRES 
evaluators  to  more  accurately  measure  and  evaluate  a  given 
mission  of  a  unit. 
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Organizationally,  the  structure  of  the  MCCRES  team 
is  one  that  facilitates  that  of  command  and  control.  [MCO 
3501.2,  1977]  It  typically  reflects  that  of  a  line  and 
staff  structure,  as  depicted  in  Figure  2.2. 


Figure  2.2  MCCRES  Organizational  Structure 
[MCO  3501.2,  1977] 

2 .  MCCRES  Process 

The  process  element  of  MCCRES,  as  an  evaluation 
system,  is  the  action  that  provides,  once  initiated,  for  the 
continuous  collecting  of  proper  information,  review  of  data, 
and  feedback  of  information,  that  is  germane  to  unit  readi¬ 
ness.  Figure  3.3,  Chapter  III,  depicts  the  MCCRES  manage¬ 
ment  control  process. 

The  MCCRES  process  begins  prior  to  the  actual  simu¬ 
lated  combat  scenario.  As  provided  in  Volume  I  of  the 
Order,  a  MCCRES  evaluation  team  is  selected  and  schooled  on 
doctrine  that  will  be  applicable  to  the  scenario  to  be  used 


in  the  unit's  MCCRES.  The  scenario,  potential  evaluators, 
schooling  of  the  evaluators,  and  the  selection  of  MPS's  are 
controlled  by  the  Headquarters  that  is  usually  two  command 
levels  above  that  of  the  unit  to  be  evaluated.  Once  the 
administering  of  the  MCCRES  evaluation  begins,  the  responsi¬ 
bility  for  the  conduct  of  the  MCCRES  is  turned  over  to  the 
Tactical  Exercise  Controller  (TEC) . 

The  scenario  is  coordinated  and  controlled  by  the 
Tactical  Exercise  Controller  (TEC)  such  that  it  allows  for 
maximum  evaluator  observation  at  the  subordinate  unit  level 
of  the  unit  receiving  the  MCCRES  evaluation  (that  is,  com¬ 
panies  are  observed  if  the  Battalion  is  the  designated  unit 
receiving  a  MCCRES) .  At  this  point  in  the  MCCRES  the  evalua 
tion  is  decentralized  to  the  unit  level  where  observation 
and  grading  takes  place.  However,  the  evaluators  meet  daily 
with  the  Senior  Evaluator  to  provide  their  notes  and  graded 
input  on  observed  tasks  at  the  subunit  level.  Upon  the  com¬ 
pletion  of  a  MCCRES  evaluation,  which  usually  takes  about 
four  days,  the  yes/no  scores  of  all  evaluator  observed  tasks 
are  aggregated  and  used  to  determine  the  units  overall  "go/ 
no  go"  grade  for  a  given  MPS.  The  total  of  all  graded 
MPS's  in  turn,  result  in  the  overall  unit  MCCRES  "score." 
[MCO  3501.2,  1977] 

Upon  completion  of  the  MCCRES,  the  unit  receives  a 
debrief  by  the  Senior  Evaluator  and  within  10  days  an  Ini¬ 
tial  Report  (by  message)  reflecting  the  MCCRES  results  is 


sent  by  the  Senior  Evaluator  to  Headquarters,  Marine  Corps. 

A  second  Follow-up  Report,  which  is  more  comprehensive  than 
the  Initial  Report  is  sent  within  thirty  working  days. 

The  entire  process  is  structured  such  that  it  is  to 
be  confidential  in  nature  and  the  information  from  the 
MCCRES  is  not  to  be  used  by  others  in  comparing  results  with 
sister  units.  The  confidentiality  and  control  of  informa¬ 
tion  helps  to  negate  the  "report  card"  effect  on  the  com¬ 
mander  of  the  unit  receiving  the  MCCRES. 

E.  SUMMARY 

The  MCCRES  is  a  system  to  evaluate  a  combat  unit's 
ability  to  perform  its  stated  mission.  The  MCCRES  Order 
sets  up  a  structure  and  process  for  the  operational  control 
of  the  evaluation  system  that  is  used  to  evaluate  whether  a 
combat  unit  is  ready  or  not  to  perform  its  stated  mission. 
These  two  elements,  structure  and  process,  are  common  sub¬ 
sets  of  any  management  control  system.  It  is  this  formula¬ 
tion  of  a  structure  and  process  to  insure  that  resources 
are  used  effectively  and  efficiently  to  meet  Marine  Corps 
objectives.  These  objectives  should  drive  the  selection, 
schooling,  and  ultimately  the  effectiveness  of  MCCRES 
evaluators.  The  evaluators  significantly  influence  the 
input  of  information,  valid  or  invalid.  Such  information 
either  contributes  to  making  the  system  perform  as  designed 
or  completely  destroys  its  credibility.  As  stated  in 
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Chapter  I,  this  paper  focuses  on  these  two  control  elements 
to  determine  if  there  might  be  a  way  to  facilitate  the  effi 
cient  and  effective  use  of  evaluators. 

Before  this  determination  can  be  done,  one  must  estab¬ 
lish  a  common  ground  as  to  what  a  management  control  system 
is,  and  what  it  is  supposed  to  accomplish.  Therefore,  the 
theory  of  management  control  is  presented  in  Chapter  III. 
Chapter  IV  presents  data  on  MCCRES  evaluators  obtained  from 
field  work  and  appropriate  analysis.  Chapter  V  pursues  the 
possibility  of  constructing  a  model  to  assist  in  the  com¬ 
mander's  selection  of  potential  evaluators.  The  model  is 
then  reviewed  to  determine  if  it  holds  up  under  previously 
stated  management  control  theory. 


III.  MANAGEMENT  CONTROL 


A.  GENERAL 

Management  Control  Theory  should  be  one  of  man's  best 
friends,  so  state  many  authors  of  writings  found  on  the 
field  of  management.  [Anderson  and  Herzlinger,  1980;  Ander¬ 
son  and  Dearden,  1980;  Bonini,  Jaeaicke,  and  Wagner,  1964; 
Roth,  Allen,  and  Smith,  1982]  This  chapter  discusses  how 
management  control  is  germane  to  the  question  posed  in  this 
thesis.  Specifically  this  chapter  discusses  management  con¬ 
trol  and  its  critical  role  in  providing  a  structure  and 
process  that  facilitates  for  the  effective  use  of  MCCRES 
evaluators  and  their  measuring  of  unit  readiness. 

The  approach  used  in  presenting  management  control  is 
to  start  with  a  working  definition  of  management  control,  to 
introduce  control  as  a  general  concept,  and  then  to  expand 
the  discussion  of  management  control  to  its  elements  of 
structure  and  process.  Ultimately,  these  concepts  collec¬ 
tively  should  bring  the  reader  to  understand  what  management 
control  systems  are  and  what  they  should  do.  Once  a  view  of 
the  management  control  system  is  presented,  an  investigation 
of  how  it  provides  for  evaluation  and  its  evaluators  is 
offered.  Finally,  the  element  of  measurement,  as  it  relates 
to  the  management  control  system,  is  discussed  in  terms  of 
its  effect  on  evaluators  in  performing  their  duties. 


21 


B.  DEFINITION 


Any  management  control  system  is  actually  a  collection 
of  integrated  control  subsystems  or  building  blocks  that 
together  provide  the  formal  means  by  which  top  management 
actually  runs  their  organization.  Some  examples  of  control 
subsystems  are: 

- Organizational  structure 

- Measures  of  performance 

- Flar.ning  and  budgeting 

- Capital,  or  program  budgeting 

- Managerial  rewards  and  punishment.  [Rotch,  1982] 

The  control  subsystems  that  directly  influence  the  investi¬ 
gation  of  this  thesis  are:  (1)  Organizational  Structure  and 

(2)  Measuring  Performance. 

As  a  common  point  of  departure,  a  working  definition  for 
management  control  needs  to  be  developed.  Unfortunately, 
there  are  as  many  definitions  for  management  control  as 
there  are  authors  in  the  field.  Possibly  the  best  known 
definition  of  management  control  is  "the  process  by  which 
managers  assure  that  resources  are  obtained  and  used  effec¬ 
tively  and  efficiently  in  the  accomplishment  of  the  organi¬ 
zation's  objectives."  [Anthony  and  Herzlinger,  1975]  This 
definition  stresses  the  accomplishment  of  organizational 
goals  through  implemented  strategies.  Another  useful  defi¬ 
nition  is  by  Hofstede,  who  defines  management  control  "as  a 
pragmatic  concern  for  results,  obtained  through  people." 


This  definition  is  useful  because  it  focuses  on  managers 
where  they  live — managing  people  as  one  of  several  available 
but  limited  resources  to  ultimately  achieve  results.  A 
lesser  known,  but  equally  descriptive  definition  is,  "the 
process  whereby  managers  lead  and  motivate  self-directed 
efforts  of  organizational  members  to  jointly  accomplish 
organizational  and  individual  goals,  using  periodic  evalua¬ 
tion  of  performance."  [Ramanathan,  1982]  Finally,  a  very 
general  definition,  "a  system  whose  purpose  is  to  attain  and 
maintain  a  desired  state  or  condition."  [Anthony,  and  Dear- 
den,  1980]  Although  the  list  of  management  control  defini¬ 
tions  could  continue,  [Anthony  and  Herzlinger,  1980;  Schc- 
derbek,  and  Kefalas,  1980;  Bonini,  Jaedicke,  and  Wagner, 
1964;  Rotch,  Allen,  and  Smith,  1982]  each  has  a  central 
theme  that  managers  of  resources  must  apply  checks  and  bal¬ 
ances  to  the  use  of  those  resources  in  order  to  achieve 
stated  goals  as  efficiently  and  effectively  as  possible. 

It  is  important  to  note  that  in  many  readings  on  manage¬ 
ment  control  the  definitions  of  the  terms  objective,  and 
goal  are  interchanged.  For  the  purpose  of  this  thesis  the 
definitions  used  will  be  that  of  Anthony  and  Herzlinger. 

They  are: 

- G°al  “  Goals  of  an  organization  are  set  in  the  stra¬ 
tegic  planning  process  and  are  broad  fairly  timeless  state¬ 
ments.  For  the  purpose  of  management  control  these  goals 


are  taken  as  given.  Management  control  is  intended  cc 
facilitate  the  achievement  of  these  goals. 

- Objective  -  Objectives  of  an  organization  are  more 

specific  statements  with  their  achievement  contemplated 
within  a  specific  time  period.  It  is  through  the  achieving 
of  these  objectives  that  an  organization  approaches  its 
attainment  of  a  stated  goal.  [Anthony  and  Herzlinger,  1930] 

C.  CONTROL 

Control  is  one  of  the  five  basic  management  processes. 
The  other  four  are,  planning,  organizing,  staffing,  and 
directing.  [Anthony  and  Herzlinger,  1980] 

1 .  Concept 

The  concept  of  control  is  one  that  is  basic  in  the 
lives  of  managers.  Control  is  the  process  by  which  actual 
output  is  compared  to  planned  output  and  the  corrections 
required  to  bring  planned  and  actual  output  closer  together 
are  accomplished.  In  ether  words,  control  is  the  process 
of  monitoring  activities  and  feeding  back  those  results  for 
the  issuance  of  further  guidance.  An  Input/Process/Output 
model,  reflecting  a  "closed  loop,"  for  the  process  of  this 
concept  is  shown  in  Figure  3.1. 
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Figure  3.1  Control  ar.d  Fee  abac'; 

[Koontz,  O'Donnell  and  Weihrich,  i 3 3 1 1 

It  is  this  decree  of  applied  control  that  affect = 
human  behavior  by  reducing  the  freedom  of  action  of  those 
managers  under  its  power.  Stated  another  way , --control  i 
result  of  a  conscious  management  process,  planning,  that 
measures  results  against  planned  effort.  The  control  sys 
must  be  compatible  with  the  goals,  design,  and  objectives 
the  organization.  In  the  planning  process,  management 
decides  what  the  organization  should  be  doing,  and  the  cc 
trol  process  compares  actual  accomplishments  with  these 
plans.  [Anthony  and  Dearden,  1980]  Thus,  in  an  organiza 
there  is  a  close  connection  between  the  planning  orocess 
the  control  process.  This  connection  is  so  close  that  fc 
many  purposes  planning  and  control  should  be  viewed  as  a 
single  process.  [Anthony  and  Dearden,  1980] 
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Bozin,  (1981)  discussed  this  tie  between  control  and 
planning  as  presented  by  Koontz  and  O'Donnell.  They  state 
that  the  basic  control  process,  regardless  of  where  it  exists 
or  what  it  controls,  can  be  seen  as  involving  three  steps: 

(1)  establishing  standards,  (2)  measuring  performance  against 
these  standards,  and  (3)  correcting  deviations  from  standards 
and  plans.  Koontz  and  O'Donnell  further  state  that  central 
to  this  view  is  the  concept  of  information  feedback.  Feed¬ 
back  is  the  process  which  discloses  errors  or  deficiencies 
in  goal  attainment  and  returns  feedback  information  to  the 
system.  Examples  of  this  control  through  feedback  are  ubiq¬ 
uitous.  They  include:  (1)  the  regulation  of  temperature 

and  respiratory  functions  in  the  human  body,  (2)  the  regula¬ 
tion  of  a  simple  mechanical  engine's  speed  through  a  system 
of  flyweights,  and  (3)  the  regulation  of  heme  heating  and 
cooling  through  a  thermostat.  Each  of  these  examples  can  be 
followed  through  the  control  and  feedback  process  presented 
in  Figure  3.1. 

2 .  Design 

Newman  (1975)  presents  control  design  as  a  series  cf 
elements.  First,  if  controls  are  to  work,  desired  results 
must  be  defined  in  measurable  terms  and  linked  to  results 
attributable  to  specific  individuals.  Secondly,  effective 
control  is  largely  based  upon  predictions  of  results  rather 
than  upon  actual  results.  Therefore,  the  designer  must 
establish  whether  the  predictors  of  results  can  be  identified 


early  in  the  process.  This  view  argues  for  the  use  of  con¬ 
trols  to  be  used  to  maintain  the  direction  of  purposeful 
behavior.  Newman  lists  predictors  of  results  as:  measure¬ 
ments  of  inputs,  success  of  early  steps,  monitoring  of  proc¬ 
ess  variables,  the  existence  of  symptomatic  conditions,  and 
relative  deviation  from  assumed  operating  conditions. 

Newman's  third  step  is  to  select  composite  feedback. 
Selecting  composite  feedback  means  selecting  the  predictors 
that  are  useful  within  the  given  system. 

As  the  fourth  step,  it  is  necessary  to  set  some  par 
value  or  standard  for  each  predictor  or  desired  result. 
Though  the  end  results  are  identified  for  the  predictor, 
there  still  is  no  way  of  knowing  whether  that  result  is  good 
or  bad  without  a  par  value. 

The  fifth  step  of  Newman's  concept  is  one  of  asking 
the  question  as  to  what  should  be  done  with  the  collected 
information.  What  should  be  reported?  To  whom  should  the 
information  be  reported?  When  should  the  information  be 
reported?  Hew  should  the  information  be  reported?  All 
these  questions  need  to  be  asked,  remembering  that  the  con¬ 
trol  information  should  be  part  of  a  formal  reporting  system 

Newman  concludes  that  after  these  five  steps  are 
accomplished  the  final,  or  sixth  step  is  to  evaluate  and 
take  corrective  action.  [Newman,  1975]  Newman's  presenta¬ 
tion  falls  entirely  within  the  process  reflected  in  Figure 
3.1,  and  follows  other  accepted  concepts  of  control  design 


[Suchman,  1967;  Weiss,  1972;  Bonini,  Jaedicke,  and  Wagner, 
1964]  . 


D .  MANAGEMENT  CONTROL 

Anthony  and  Herzlinger,  (1980)  state  planning  and  control 
cannot  be  separated  or  even  distinguished  as  separate  enti¬ 
ties  in  most  cases.  They  further  stipulate  the  combination 
of  these  two  processes  can  be  divided  into  three  other  dis¬ 
tinct  processes:  (1)  strategic  planning,  (2)  management  con¬ 
trol,  and  (3)  operational  control.  These  three  processes, 
their  definitions,  and  relationship  to  each  other  will  be 
discussed  in  this  section. 

Anthony  and  Herzlinger,  (1983)  in  their  discussion  on 
management  control,  state  there  are  two  important  activities 
that  all  managers  engage  in:  (1)  planning  and  (2)  control . 
Planning  is  deciding  what  should  be  done  and  how  it  should 
be  done,  and  control  is  assuring  that  the  desired  results 
are  obtained.  In  most  organizations,  three  different  types 
of  planning  and  control  processes  can  be  identified:  (1) 
strategic  planning,  (2)  management  control,  and  (3)  opera¬ 
tional  control.  Their  definitions  of  these  processes  are: 

- Strategic  Planning  -  Strategic  Planning  is  the  process 

of  deciding  on  the  goals  of  the  organization  and  on  the  broad 
strategies  that  are  to  be  used  in  attaining  these  goals. 

- Management  Control  -  Management  Control  is  the  process 

by  which  management  assures  that  the  organization  carries  out 


its  strategies,  through  the  use  of  objectives,  as  effec¬ 
tively  and  efficiently  as  is  possible. 

- Operational  Control  -  Operational  Control  is  the 

process  of  assuring  that  specific  tasks,  in  support  of 
established  objectives,  are  carried  out  effectively  and 
efficiently. 

These  three  processes  blend  into  one  another  and  do  not 
necessarily  have  sharp,  well  defined  lines.  However,  Anthony 
and  Herzlinger  (1980)  argue  that  strategic  planning  sets  the 
guidelines  for  management  control,  and  management  control 
sets  the  guidelines  for  operational  control.  The  complete 
management  function  involves  an  integration  of  all  these 
processes,  and  the  processes  are  complementary. 

1 .  Management  Control  Structure 

Management  control  sets  guidelines  for  operational 
control.  The  structure  of  management  control  is  used  to 
delegate  responsibility  and  assign  appropriate  authority  for 
the  performance  of  specific  duties.  The  structure  of  opera¬ 
tional  control  is  used  to  take  those  specific  duties  and 
break  them  down  further  into  individual  tasks  such  that  it 
allows  the  manager  to  apply  sufficient  control  to  accomplish 
each  task.  A  model  reflecting  the  structure  of  management 
control  as  it  pertains  to  MCCRES  is  provided  from  the  MCCRES 
Order  and  is  depicted  in  Figure  3.2.  It  presents  the  MCCRES 
organizational  structure  for  required  authority/responsi¬ 
bility  of  those  assigned  billets  in  the  performance  of  their 
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mission  (goal) .  An  example  of  the  structure  of  operational 
control  is  the  breaking  down  of  MPS 1 s  into  separate  task, 
requirements,  and  conditions,  as  reflected  in  Figure  2.1. 


Figure  3.2  MCCRES  Management  Control  Structure  Model 

The  use  of  the  model  in  Figure  3.2  is  an  example  of 
the  established  "top  down"  flow  of  authority  required  by 
most  traditional  line/staff  organizations  to  communicate 
their  goals,  objectives,  and  operational  requirements  from 
"top  management,"  to  their  subordinate  managers.  It  is  the 
organization  of  this  successive  layering  of  authority  that 
produces  a  system  for  the  organization  to  accomplish  its 
purpose.  Management  control  structure  effectively  facili¬ 
tates  the  delegation  of  commensurate  power/authority  for 
each  level  of  management  to  insure  that  proper  control  is 
executed  at  a  given  level  so  that  objectives  and  goals  are 


achieved.  The  model  reflected  in  Figure  3.2  is  discussed 
further  in  Chapter  IV,  in  the  investigation  of  how  manage¬ 
ment  control  structure  can  effect  the  proper  selection  and 
use  of  MCCRES  Evaluators. 

2 .  Management  Control  Process 

Management  control  process  are  those  actions  that 
take  place  to  accomplish  specific  goals  that  have  been  estab 
lished  by  management. 

To  model  the  management  control  process  one  must 
maintain  the  distinction  between  structure  and  process.  The 
structure  of  the  management  control  system  can  be  described 
in  terms  of  the  units  in  an  organization  and  the  nature  cf 
the  information  that  flows  among  these  units.  The-  manage¬ 
ment  control  process  is  what  the  managers  do  with  this 
information.  [Anthony  and  Dearden,  1980]  To  assist  in 
distinguishing  the  difference  between  structure  and  process, 
a  model  of  management  control  process  is  presented  in 
Figure  3.3. 
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Figure  3.3  MCCRES  Management  Control  Process  Model, 
adapted  from  Ramanathan  (1982,  p.  176) 


This  model  (Figure  3.3)  reflects  the  process  MCCRES 
managers  use  in  the  achieving  of  objectives.  It  does  not 
account  for  authority  and  responsibility  as  does  the  struc¬ 
ture  model  shown  in  Figure  3.2;  it  simply  establishes  the 
action  required  in  the  interchange,  or  communication,  of 
information  between  organizational  functional  areas.  The 
model  depicts  the  mission  of  MCCRES,  which  is  developed 
through  strategic  planning,  and  provides  a  means  to  monitor 
desired  goals.  Specific  objectives  are  then  developed  that 
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can  be  accomplished  through  a  given  program.  This  program 
is  funded  through  periodic  budgeting  which  allows  for  the 
evaluation  and  reporting  of  MCCRES  results.  Evaluator  per¬ 
ception,  preference,  and  acceptance  has  a  significant  effect 
directly  on  the  evaluation  and  reporting  phase  and,  indi¬ 
rectly,  on  the  entire  model.  Further  discussion  of  this 
model  and  how  it  affects  the  development  of  the  answer  to  the 
question  posed  in  this  thesis  will  be  pursued  in  Chapter  IV. 

Anthony  and  Ker2linger  (1980)  present  another  view  of 
the  management  control  process  which  can  be  imposed  on  the 
MCCRES  management  control  process  model.  Figure  3.3,  without 
distorting  it.  They  describe  management  control  as  something 
that  takes  place  in  an  organization  that  already  exists,  that 
has  goals,  and  that  has  decided  on  broad  strategies  for 
achieving  these  goals.  Decisions  on  these  goals  and  strate¬ 
gies  are  made  in  the  strategic  planning  process.  Anthony 
and  Herzlinger  state  that  this  process  is  largely  unsystem¬ 
atic  and  informal.  The  management  control  system  collects 
information  that  is  useful  in  strategic  planning.  But,  the 
management  control  system,  in  itself,  does  not  provide  this 
information  to  managers  in  any  structured,  routine  fashion 
during  a  given  strategic  planning  session.  Rather,  it  must 
be  assembled  into  a  proper  format  when  the  need  arises,  and 
in  the  form  required  for  addressing  a  specific  strategic 
problem  or  the  restructuring  of  a  given  organizational  goal. 
This  reorganized  and  newly  formatted  information  can  then 


assist  management  in  its  resolution  of  a  given  strategic 
planning  problem. 

Anthony  and  Dearden  (1980)  ,  and  Anthony  and  Herz- 
linger  (1980),  developed  the  concept  of  management  control 
process  as  one  that  has  four  principal  steps  or  phases.  The 
acknowledge  that  there  is  an  informal  management  control  sys 
tem  that  consists  of  information  flowing  between  managers 
through  the  use  of  meetings,  conversations,  and  even  facial 
expressions.  But  this  informal  system  does  not  lend  itself 
to  one  of  systematic  description. 

The  other  more  formal  management  control  system  dees 
however ,  lend  itself  to  description.  It  is  one  that  takes 
information  that  consists  of  planned  (or  estimated)  data  and 
actual  data  on  inputs  and  outputs,  and  uses  this  information 
through  reports  to  determine  how  close  actual  inputs/outputs 
are  to  meeting  the  planned  inputs /outputs ,  and  then  taking 
action  on  the  basis  of  this  information. 

Anthony,  Herzlinger,  and  Dearden  go  on  to  discuss 
the  principle  steps;  (1)  Programming,  (2)  Budgetina,  (3) 
Operating  (and  measurement) ,  and  (4)  Reporting  and  Analysis. 
They  state  it  is  the  flow,  or  process,  of  these  four  phases 
that  show  how  management  control  "closes  the  loop"  on  the 
business  of  accomplishing  objectives  in  a  more  effective  and 
efficient  way.  To  assist  in  this  discussion,  Figure  3.4 
depicts  how  each  phase  follows  the  other  and  how  the  loop  is 
continuous  in  its  process. 


Extamal 

information 


9udgaf/„g 


Extamal 

information 


Strataaiaa 


*frategia» 


^Poning  and*^ 

t 

External 

Information 


External 

Information 


Figure  3.4  Phases  of  Management  Control 
[Anthony,  Dearden,  1980,  and  Anthony,  Herzlinger,  1980] 


3.  Phases  of  Manaaement  Control  Process 


In  order  to  follow  the  illustration  depicted  in 
Figure  3.4,  each  phase  is  presented  as  defined  by  Anthony, 
Herzlinger,  and  Dearden. 

- Programming  -  In  the  programming  phase,  decisions 

are  made  with  respect  to  the  major  programs  the  organiza¬ 
tion  plans  to  undertake  during  the  coming  period.  These 
decisions  either  are  made  within  the  context  of  the  goals 


and  strategies  that  have  previously  been  decided  upon,  or 
they  represent  changes  in  strategy. 

- Budgeting  -  A  budget  is  a  plan  expressed  in  quan¬ 
titative,  usually  monetary,  terms  covering  a  specified 
period  of  time.  The  agreed-upon  budget  is  a  bilateral  com¬ 
mitment  between  superiors  and  their  subordinates. 

- Operating  and  Measurement  -  During  the  period  of 

actual  operations,  records  are  kept  of  resources  actually 
consumed  and  outputs  actually  achieved.  The  records  of 
resources  consumed  are  structured  so  that  costs  are  col¬ 
lected  both  by  programs  and  by  subordinate  cost/responsi- 
bility  centers. 

- Reporting  and  Analysis  -  Accounting  information 

along  with  a  variety  of  other  information  is  summarized, 
analyzed,  and  reported  to  those  who  are  responsible  for 
knowing  what  is  happening  in  the  organization,  and  for 
improving  performance.  These  reports  compare  planned 
inputs/outputs  with  actual  inputs/outputs.  These  reports 
are  used  to  coordinate  and  control  current  activities, 
evaluate  operating  performance,  and  are  used  as  a  basis  for 
program  evaluation. 

E.  MANAGEMENT  CONTROL  SYSTEM 

The  management  control  system  consists  of  a  structure 


and  a  process;  that  is,  what  it  is  and  what  it  does.  As  has 
been  previously  presented,  the  structure  of  a  management 


control  system  can  be  described  in  terms  of  the  units  in  an 
organization  and  the  nature  of  the  information  that  flows 
among  these  units.  The  process  is  what  the  managers  do  with 
this  information.  [Anthony  and  Herzlinger,  1980] 

The  MCCRES  management  control  structure  model.  Figure  3. 
is  an  example  of  a  management  control  structure  that  contrib 
utes  as  a  subsystem  to  that  of  a  larger  control  system,  that 
of  the  Marine  Corps'  management  control  system.  Figure  3.3 
reflects  a  management  control  process  that  pertains  directly 
to  that  of  MCCRES,  but  also  falls  within  the  category  of  a 
contributing  control  subsystem  to  that  of  the  Marine  Corps' 
overall  management  control  process.  It  is  the  combination 
of  these  two  elements  of  the  MCCRES  control  subsystem,  and 
many  other  separate,  mutually  contributing  subsystems,  that 
provide  the  overall  management  control  system  of  the  Marine 
Corps.  MCCRES,  by  providing  feedback,  contributes  to  the 
overall  control  of  the  United  States  Marine  Corps. 

F.  MANAGEMENT  CONTROL  SYSTEM  CHARACTERISTICS 

The  management  control  system  is  a  total  system  com¬ 
prised  of  many  singular  systems.  In  describing  the  manage¬ 
ment  control  system,  its  structure  and  process,  there  are 
five  characteristics  that  influence  its  effectiveness: 

(1)  total  system,  (2)  goal  congruence,  (3)  financial  frame¬ 
work,  (4)  rhythm,  and  (5)  integration.  It  will  be  helpful 
to  examine  these  characteristics  to  more  fully  understand 
management  control  systems.  [Anthony  and  Herzlinger,  1980] 
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Anthony  and  Herzlinger  state  the  characteristic  that  pre¬ 
sents  the  management  control  system  as  a  total  system  is  most 
important.  They  stress  it  must  operate  as  such  in  order  to 
ensure  a  proper  balance  between  all  functioning  parts  of  the 
operation.  In  order  to  maintain  this  balance,  management 
must  have  access  to  information  from  each  of  its  parts. 

The  second  characteristic  is  that  of  goal  congruence. 
Anthony  and  Herzlinger  state  that  if  an  organization  designs 
its  management  control  system  so  that  the  actions  it  leads 
its  managers  to  take  are  in  accordance  with  the  manager's 
own  self-interests,  but  will  also  be  actions  that  are  in  the 
best  interest  of  the  organization,  there  is  goal  congruence. 
This  type  of  design  provides  mutually  supporting  goals  that 
are  in  congruence  with  cne  another.  That  is  to  say,  given 
goal  congruence  has  been  developed,  the  personal  goals  cf 
people  in  the  organization  are,  at  least  somewhat,  consist¬ 
ent  with  the  designed  goals  cf  the  organization  as  a  whole. 

The  third  characteristic  of  the  management  control  sys¬ 
tem  is,  with  rare  exception,  that  it  is  built  around  a 
financial  framework.  That  is,  the  system  is  built  in  a  way 
which  facilitates  the  measuring  of  all  resources  in  a  given 
standard,  usually  monetary  units.  Anthony  and  Herzlinger 
point  out  that  money  is  the  only  common  denominator  that 
provides  a  heterogeneous  metering  of  the  mixture  of  resources 
used  in  the  operation  of  running  an  organization.  Because 


cf  this,  a  mixture  of  resources  can  be  combined  and  cimp... 
as  elements  of  input  and  products  of  output. 

The  fourth  character istic  of  management  control  is  the 
of  rhythm.  Anthony  and  Herzlinger  state  than  management 
tends  to  be  rhythmic;  it  follows  a  definite  pattern  and  :: 
table,  month  after  month,  and  year  after  year.  acause  of 
this  characteristic  there  tends  to  be  ample  evidence  that 
certain  organizational  tasks  are  predictable.  [Anther.-/  nr. 
Herzlinger,  1980;  Anthony  and  Dear den,  1930]  This  charac¬ 
teristic  should  be  of  considerable  help  ir.  the  business  ;f 
managing  resources  so  that  increased  efficiency  and  effec¬ 
tiveness  are  realistic  goals. 

Lastly,  Anthony  and  Herzlinger  state  that  a  management 
control  system  should  be  a  coordinated,  integrated  system.: 
all  data  collected,  regardless  of  its  primary  purpose,  mus 
be  reconcilable  with  one  another. 

G.  EVALUATION 

Evaluation  is  a  critical  process  that  takes  place  in  a 
management  control  system.  This  process  is  one  of  gather! 
information  required  by  management  on  which  to  base  decisi 
that  will  keep  the  organization's  objectives  in  proper  bal 
ance  with  that  of  organizational  goals.  Evaluation  takes 
data  that  resides  within  a  system  and  provides  a  means  for 
feedback  of  perspicuous  information  pertinent  to  that  of 
management  control. 


First,  a  definition  for  evaluation  is  offered: 

"Evaluation  is  the  process  of  determining  the  value  or 
amount  of  success  in  achieving  a  predetermined  objective. 

It  includes  at  least  the  following  steps:  Formulation  of 
the  objective,  identification  of  the  proper  criteria  to 
be  used  in  measuring  success,  determination  and  explana¬ 
tion  of  the  degree  of  success,  and  recommendations  for 
further  program  activity."  [Suchman,  1967,  p.  23] 

Stufflebeam,  (1971)  defines  evaluation  as  "the  process 
of  delineating,  obtaining,  and  providing  useful  information 
for  judging  decision  alternatives."  Wheeler,  (19S3)  points 
out  that  there  seems  to  be  two  common  factors  to  all  defini¬ 
tions  on  evaluation.  The  first,  evaluation  is  concerned 
with  making  a  judgement  or  assessment  about  something.  And 
secondly,  that  judgement  can  be  made  in  terms  of  some  objec¬ 
tive  or  goal.  The  purpose  of  evaluation  seems  not  to  be  that: 
of  proving  but  rather  to  that  of  improving.  However,  evalua¬ 
tion  itself  is  not  an  end  in  itself,  it  is  only  a  tool  that 
can  be  used  to  contribute  to  decisions.  [Stufflebeam,  1971] 
Given  this,  Wheeler  states  that  one  should,  "look  at  evalua¬ 
tion  as  a  judgement  of  some  program  with  the  purpose  of  con¬ 
tributing  to  decisions  concerning  the  current  attainment  of 
that  program's  objectives  or  goals."  [Wheeler,  1983] 

In  order  to  better  understand  the  evaluation  process, 
one  should  be  acquainted  with  the  characteristics  and  types 
or  levels  of  evaluation. 

1 .  Evaluation  Characteristics 

Wheeler,  (1983)  provides  a  review  of  the  current 
writings  on  the  subject  of  evaluation  characteristics.  His 
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presentation  is  not  intended  to  be  comprehensive,  but  does 
provide  the  reader  with  an  appreciation  for  what  evaluation 
must  do. 


Evaluation  should: 

1.  Be  conducted  in  terms  of  purpose.  That  is,  the  objec¬ 
tives  must  be  known.  If  the  objectives  are  not  known,  the 
evaluation  effort  cannot  measure  how  well  they  are  being 
attained. 

2.  Be  cooperative.  Cooperation  at  all  organizational  levels 
is  essential.  Without  free  communication,  evaluation  results 
will  not  reach  all  parties,  hence  diluting  their  usefulness. 

3.  Be  continuous.  Evaluation  muse  be  an  on-going  orocess 
to  accurately  track  performance  and  aid  planning  in  light  of 
current  objective  attainment. 

4.  Be  specific.  leneralizations  are  not  as  useful  as  spe¬ 
cific  information  in  providing  performance  information. 

5 .  Provide  means  and  focus  to  appraise  self,  practice,  and 
product .  The  evaluation  must  provide  information  of  suffi¬ 
cient  guality,  and  specificity,  to  evaluate  not  only  the 
program  output,  but  the  mechanism  of  converting  inputs  to 
output  and  the  individuals'  performance  within  the  mechanism. 

6 .  Be  based  cn  uniform  and  objective  methods  and  standards. 
Methods  and  standards  which  change  from  one  evaluation  to 
the  next  destroy  trust  and  leave  these  being  evaluated  ques¬ 
tioning  how  they  should  oerform  their  work  tasks.  fWheeler, 
1983] 

2 .  Evaluation  Types 


Suchman,  (1967)  defines  the  five  types  or  levels  of 
evaluation:  (1)  effort,  (2)  adequacy,  (3)  process,  (4) 

effectiveness,  and  (5)  efficiency.  Stufflebeam,  et  al. 
(1971)  on  the  other  hand  defines  four  types  of  evaluation: 
(1)  context,  (2)  input,  (3)  process,  and  (4)  product.  To 
compare  the  differences  between  the  two  authors  and  their 
definitions  of  evaluation  types,  or  levels,  a  short  discus¬ 
sion  of  each  definition  follows. 
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Effort - Effort  is  equivalent  to  input,  so  an 


evaluation  that  uses  effort  as  the  measure  of  performance  is 
measuring  input  values  as  indicators  of  meeting  objectives, 
i.e.,  how  much  money  was  spent  or  how  many  man-hours  were 
used.  However,  this  use  of  inputs  or  resources  may  or  may 
not  mean  that  the  job  is  being  accomplished. 

Effectiveness - If  inputs  are  too  far  removed  from 

the  meeting  of  organizational  objectives,  why  not  look  at 
output?  Evaluating  outputs  could  eliminate  the  problem  of 
input  measurements.  Although  effectiveness  is  an  arbitrary 
definition  it  does  structure  a  process  for  comparison  of 
output  against  organizational  objectives. 

Adequacy - Adequacy,  or  impact,  looks  at  performance 

in  terms  of  its  larger  environment.  In  other  words,  it  is 
an  output- tc-need  relationship.  However,  the  problem  in  any 
impact  or  adequacy  level  of  evaluation  is  the  problem  of 
identifying  the  overall  need. 

Efficiency - Efficiency  is  another  level  of  evalua¬ 

tion  that,  in  some  ways,  overcomes  the  shortcomings  of  the 
previous  levels.  Efficiency  is  probably  the  most  familiar 
level  of  evaluation.  Efficiency  relates  output  to  input. 

In  terms  of  efficiency,  things  are  better  if  more  can  be 
done  with  the  same  amount  of  inpu  jr  the  same  output  can  be 
generated  with  less  input. 

Process - Here  process  is  defined  as  the  relation¬ 

ship,  or  function,  between  input  and  output.  Process 
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evaluation  attempts  to  focus  on  the  mechanism  by  which 
effort  is  translated  into  output.  In  other  words,  output 
is  viewed  as  a  function  of  effort.  The  function  assumes  an 
understanding  of  how  the  organization  operates  and  an 
ability  to  predict  what  the  output  of  the  oraanization  will 
be  for  a  particular  input.  It  is  this  view  of  process 
evaluation  that  allows  a  manager  to  observe  the  entire 
transformation,  starting  with  raw  input  and  ending  with  the 
output  necessary  for  meeting  organizational  goals. 

Context  Evaluation - Context  evaluation  is  used  in 

the  process  of  the  planning  decision  in  determining  the 
goals  and  objectives.  Context  evaluation  aids  the  planning 
decision  in  the  diagnosing  of  problems  and  identifying 
objectives.  There  are  two  types  of  context  evaluation: 
contingency  and  congruence.  Contingency  evaluation  is  used 
to  look  across  the  boundary  of  the  system  of  interest  and 
ask  what-if  kinds  of  questions.  Congruency  evaluation  takes 
the  environment  and  resources  availability  as  given  and  then 
asks  questions  about  how  a  particular  goal  or  objective  will 
be  met. 

Input  Evaluation - Once  the  goals  are  decided  upon, 

then  input  evaluation  must  be  accomplished.  Input  evalua¬ 
tion  is  useful  in  determining  the  structuring  of  decisions 
for  project  design.  Input  evaluation  is  concerned  with  the 
question  of  resource  availability,  i.e.,  how  should  the 
process  be  structured  to  utilize  the  resources? 


Process  Evaluation - Cnee  the  design  is  decided, 

process  evaluation  is  conducted.  Process  evaluation  is  used 
in  implementing  and  controlling  project  operations.  As  the 
name  implies,  process  evaluation  is  the  analysis  of  the 
process  as  developed  so  than  procedural  problems  can  be 
identified.  Changes  in  the  process  can  then  be  made.  Addi¬ 
tionally,  process  evaluation  maintains  a  record  of  what  is 
hapDening . 

Product  Evaluation - Product  evaluation  focuses  upon 

output.  Product  evaluation  is  used  in  recycling  decisions 
to  judge  and  react  to  attainments.  Product  evaluation  can 
be  viewed  as  the  decision  to  adjust  the  system  in  post-action 
control.  [Suchman,  1967;  Stuff lebeam,  1971;  Euske ,  in  press] 

Each  of  the  two  presentations  of  types,  or  categories 
of  evaluation,  have  similarities  but  are  approached  from 
different  perspectives.  Stuff lebeam  et  al.  is  concerned 
with  when  to  evaluate,  and  views  it  occurring  at  different 
stages.  Suchman  is  concerned  with  what  to  evaluate  at  a 
given  time.  [Euske,  in  press]  However,  it  is  possible  to 
effectively  combine  these  two  concepts  on  evaluation.  This 
will  be  further  discussed  in  Chapter  IV. 

H .  EVALUATORS 

Evaluators  gather  a  type  of  information  that  is  the 
heart  of  management  control — measurement  indicators.  How 
well  evaluators  do  this  is  critical  to  the  proper  functioning 


of  the  feedback  system  that  provides  information  required  by 
any  control  system.  [Wheeler,  1983]  It  is  this  information 
which  is  collected  and  in  turn  becomes  input  to  that  of  a 
larger  system:  The  Management  Control  System.  If  this  is 
done  poorly,  then  a  predictable  result  is  attained.  Ir  is 
described  by  that  tired,  but  true  cliche,  "garbage  in,  gar¬ 
bage  out .  " 


1 .  Relationship  to  the  System 

If  the  input/process/output  model,  as  reflected  in 
Figure  3.1,  is  modified  to  show  a  "closed  loop"  model  with 
a  feedback  process  added  and  with  evaluators  contributing 
information  to  that  feedback  process,  then  one  can  see  how 
critical  evaluators  are  to  the  proper  functioning  of  the 
overall  system.  Figure  3.5  provides  a  model  for  this 
discussion . 


Input 


•Process' 


►Output 


Evaluation  systems  are  the  result  of  much  concern 
and  of  special  design  to  insure,  to  the  best  extent  possible, 
that  the  measurement  indicators  are:  (1)  valid,  (2)  objec¬ 
tive,  and  (3)  reliable.  [Weiss,  1972;  Ivancevich,  1977] 
Evaluators  must  maintain  sufficient  distance,  or  autonomy, 
from  what  they  are  evaluating  to  be  effective.  Wheeler, 

(1983)  in  his  discussion  on  evaluators  and  their  relation¬ 
ship  with  objectivity,  reliability,  and  validity,  provides 
reasonable  definitions  that  are  a  collage  from  assorted 
readings  on  the  subject.  Each  definition  is  listed  below. 

- Objectivity  -  Objectivity,  in  the  context  of 

evaluation,  is  the  ability  to  observe  something  only  as  it 
physically  exists  without  the  inclusion  of  personal  feelings 
about  the  object. 

- Reliability  -  Reliability  is  based  on  the  ability 

to  replicate  observations.  If  a  particular  observation  of 
an  objective  can  be  replicated,  that  observation  is  assumed 
to  be  reliable. 

- Validity  -  Validity  is  critical  to  evaluation. 

If  an  observation  does  not  accurately  reflect  the  qualities 
of  an  object  one  wishes  to  measure,  a  "true"  evaluation  of 
the  object  may  be  impossible.  It  is  the  question  of,  how 
close  does  the  observation  represent  reality? 

If  evaluators  are  conscious  of,  and  strive  to  main¬ 
tain  a  proper  perspective  on  the  above  listed  elements  of 
evaluation,  they  will  be  closer  to  the  epicenter  of  what 

evaluation  should  accomplish. 
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The  issue  of  the  desired  degree  of  autonomy  for 
evaluators  must  be  considered  when  establishing  the  source 
from  which  potential  evaluators  will  be  sought.  Euske ,  (ir. 
press)  points  out  the  advantages  and  disadvantages  of  evalua¬ 
tors  selected  from  within  an  organization  versus  those  who 
are  external  to  it.  Each  has  their  strong  and  weak  points. 
Evaluators  who  come  into  an  organization  may  be  more  objec¬ 
tive  and  possess  a  higher  degree  of  autonomy,  but  they  may 
fight  a  certain  amount  of  animosity  from  those  within  the 
organization.  Conversely,  evaluators  who  come  from  within 
an  organization  may  be  less  objective  because  of  their  lack 
of  autonomy  with  the  organization.  [Herbert,  1979]  Euske ' s 
conclusion  is  that  a  solution  may  be  that  of  a  combination 
of  the  two  type-s  of  evaluators. 

As  a  conclusion  to  this  section,  the  observation  of 
an  ideal  evaluator  is  offered  by  Wheeler,  (1983)  and  defined 
by  Barrett,  "....the  ideal  evaluator  who  observes  and 
evaluates  what  is  important  and  reports  his  judgement  with¬ 
out  bias  or  appreciable  error  does  not  exist,  or  if  he  does, 
we  don't  know  how  to  separate  him  from  his  less  effective 
colleagues . " 

2 .  Evaluator  Error 

The  degree  of  evaluator  error  that  takes  place 
during  an  evaluation  can  be  the  greatest  single  adverse  con¬ 
tribution  to  a  well  designed  evaluation  system  which  has 
provided  for  valid,  objective,  and  reliable  measurements. 
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Cummings  and  Schwab  (1973),  distinguish  two  main  groups  of 
evaluator  errors.  They  are  (1)  variable  and  (2)  constant 
errors . 

The  above  authors  contend  that  the  first  main  group, 
variable  error,  is  the  result  of  evaluator  disagreement  from 
either  (1)  disagreement  between  evaluators,  or  (2)  a  single 
evaluator,  over  time. 

The  first,  disagreement  between  evaluators,  can  be 
reduced  by:  (1)  reduction  or  elimination  of  subjectivity  in 

measurement  instruments,  and  (2)  ensuring  evaluator  famil¬ 
iarity  with  the  job  being  evaluated.  The  second,  that  of 
single  evaluator  error  over  time,  results  from  disagreements 
in  evaluations  made  by  one  evaluator  at  different  points  in 
time.  It  is  the  inconsistent  application  of  a  given  stan¬ 
dard  that  produces  such  aberration  to  measurement  informa¬ 
tion.  A  possible  method  of  reducing  disagreements  over  time 
is  the  testing  of  potential  evaluators  and  choosing  those 
who  demonstrate  little  of  this  type  error.  An  example  of 
this  testing  method  would  be  to  present  a  potential  evalua¬ 
tor  with  a  given  scenario  and  require  him  to  grade  it  sev¬ 
eral  times  over  a  given  period.  The  individual  who 
consistently  applies  a  determined  standard  would  be  accepta¬ 
ble  for  selection  as  an  evaluator. 

The  second  main  group,  that  of  constant  error,  is 
somewhat  different  from  variable  error.  While  variable 
errors  create  differences  between  evaluations,  constant 


errors  tend  to  cause  spurious  similarities.  Constant  error 
takes  three  forms:  (1)  halo  effect,  (2)  central  tendency, 
and  (3)  leniency.  [Wheeler,  1983]  Their  definitions  are: 


- Halo  error  -  Halo  error  occurs  when  the  evaluator 

fails  to  differentiate  between  individual  items  or  dimen¬ 
sions  in  his  evaluation,  but  evaluates  on  the  basis  of  his 
overall  impression. 

- Central  tendency  -  Central  tendency  is  the  tend¬ 
ency  for  evaluators  to  rate  all  dimensions  of  an  object  near 
the  middle  of  the  evaluation  scale,  avoiding  the  extremes. 

- Leniency  -  This  error  is  committed  when  an  evalua¬ 
tor  tends  .3  rate  all  objects  too  high.  The  "easy  grader" 
consistently  delivers  inflated  rating  marks.  The  opposite 
error,  that  of  rating  all  objects  too  low  is  called 
strictness . 

To  help  in  the  correction  of  constant  error,  evalua¬ 
tor  training  is  a  useful  technique.  Through  training, 
evaluators  are  made  to  become  aware  of  these  shortcomings 
and  shown  methods  to  help  overcome  them. 

The  business  of  evaluator  error  and  its  effect  on  an 
evaluation  system  is  at  the  heart  of  this  thesis.  There¬ 
fore,  Chapter  IV  will  look  at  this  in  detail. 

I.  MEASUREMENT 

Measurement  is  a  critical  element  of  any  management  con¬ 
trol  system.  It  is  this  element  that  allows  evaluators  to 
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do  what  they  must  do — evaluate  performance,  effectiveness 
and  efficiency.  If  a  system  does  not  allow  for  providing 
good  measurement  parameters  it  effectively  negates  all  sin¬ 
cere  efforts  on  the  part  of  any  evaluator. 

Measurement  is  used  to  quantify  the  feedback  information 
that  is  used  in  determining  the  efficiency  cr  effectiveness 
of  a  system.  Euske  (in  press)  argues  that  measurement  is  a 
process  required  to  obtain  the  information  needed  for  carry¬ 
ing  out  management  control  functions:  planning,  control,  an 
evaluation.  The  quantitative  results  that  are  used  in 
evaluation  are  the  result  of  some  measurement  process. 

[Euske,  in  press] 

Wheeler,  (1983)  in  his  discussion  on  measurement,  pro¬ 
vides  Figure  3.6  to  illustrate  measurement  and  its  lack  of 
complete  correlation  between  that  of  a  given  construct  to  be 
measured,  and  the  measurement.  A  construct  is  defined,  for 
this  discussion,  as  an  aggregation  of  parts  or  elements  that 
form  a  particular  entity  or  svstem. 


Figure  3.6  Measurement  Deficiency  or  Contamination, 
Adapted  from  Xiimoski  (1974) 

Wheeler  states  that  the  deficiency  occurs  when  the  mea¬ 
sure  fails  to  take  into  account  all  of  the  factors  present 
in  the  construct.  For  example,  a  measure  of  a  data  process¬ 
ing  department's  performance  which  accounted  for  quantity  or 
output,  but  neglected  quality  and  timeliness,  would  probably 
be  considered  deficient.  He  goes  on  to  define  measurement 
contamination,  as  contrasted  to  measurement  deficiency. 
Measurement  contamination  occurs  when  the  measure  takes  into 
account  factors  which  fail  outside  the  construct.  He  goes 
on  to  illustrate,  "if  the  measure  of  the  data  processing 
department's  performance  includes  items  such  as  corporate 
sales  or  top  management's  perceptions  of  the  department,  the 
measure  is  likely  to  be  contaminated."  [1983,  p.  29]  It  is 
the  combination  of  measurement  deficiency  and  contamination 
that  can  adversely  affect  the  measure  of  validity.  [Wheeler 


J.  SUMMARY 


The  purpose  of  this  chapter  was  to  give  the  reader  suf¬ 
ficient  knowledge  and  an  appreciation  of  Management  Control 
Systems.  It  is  hoped  this  foundation  can  be  used  for  the 
later  presentation  of  data  and  its  analysis  to  be  presented 
in  Chapter  IV. 

First,  the  concept  of  control  was  presented  as  it 
relates  to  that  of  management  and  organizational  structure 
and  process.  Next,  the  focus  was  narrowed  to  that  of  manage¬ 
ment  control.  The  concept  of  management  control  having  both 
a  structure  and  process  was  presented  through  the  use  of 
applicable  models. 

An  examination  of  Management  Control  Systems  and  irs 
characteristics  was  presented  to  reflect  how  it  contributes 
as  an  integral  part  of  how  managers  control  their  organiza¬ 
tion's  functions  so  that  strategic  plans  are  achieved.  It 
was  demonstrated  how  this  could  be  accomplished  through  the 
use  of  specific  objectives  that  would  contribute  to  organi¬ 
zational  goals. 

Next,  a  review  of  evaluation  and  its  role  in  the  feed¬ 
back  process  of  management  control  was  discussed.  It  was 
pointed  out  that  evaluation  also  presents  definite  charac¬ 
teristics  and  distinctive  types.  A  review  of  measurement 
and  its  direct  and  influential  contribution  to  evaluation 
was  also  discussed. 
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Lastly,  the  use  of  evaluators  was  discussed, 
evaluators  fit  into  the  overall  scheme  of  a  rr.ar.ac 
trol  system,  hew  evaluators  make  errors,  and  hew 
designed  system  might  be  able  to  correct  seme  of 
errors  was  presented.  The  particular  format  usee 
presentation  of  this  chapter  was  used  because  it 
focus  on  evaluators ,  their  selection,  and  their  c 
ness  that  becomes  the  subject  of  Chapter  IV. 


IV.  THREE  CASE  STUDIES 


A.  GENERAL 

The  purpose  of  this  chapter  is  to  present  and  examine 
data  obtained  from  interviews  with  individuals  who  have  par¬ 
ticipated  as  an  Evaluator,  Senior  Evaluator,  Tactical  Exer¬ 
cise  Controller,  or  Evaluation/Exercise  Director  of  recent 
MCCRES  evaluations. 

The  relationship  between  the  data  collected  and  that  of 
accepted  management  control  theory  provides  the  basis  for 
the  development  of  a  more  effective  and  efficient:  model  that 
provides  for  the  selection  of  MCCRES  evaluators .  The  model 
is  designed  to  provide  the  unit  commander  a  means  of  using 
limited  resources  in  a  more  efficient  and  effective  way. 

This  model  is  developed  and  discussed  in  Chapter  V,  "A 
Model .  " 

This  chapter  is  presented  in  two  parts.  The  first  part 
is  the  presentation  and  discussion  of  data  collected  on  each 
of  three  case  studies.  The  second  part  of  the  chapter  is  an 
analysis  of  those  case  studies.  This  analysis  is  used  to 
compare  the  case  study  data  with  those  elements  of  manage¬ 
ment  control,  as  discussed  in  Chapter  III?  structure  and 
process.  The  purpose  of  the  analysis  is  to  determine  if 
there  are  any  management  control  features  found  in  the  col¬ 
lected  data  that  can  contribute  to  the  improvement  of  the 
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commander's  ability  to  select  more  effective  evaluators.  If 
there  are,  the  resulting  consequences  can  be  used  to  change 
the  existing  MCCRES  model  to  accommodate  the  new  structure 
or  process. 

B.  PRESENTATION  OF  CASi-iS 

MCCRES  evaluator  data  was  gathered  over  a  two  month 
period.  The  observations  of  portions  of  MCCRES ' s  and  inter¬ 
views  took  place  at  three  separate  major  commands,  each  with 
a  distinctly  different  combat  mission:  (1)  A  Fixed  Wing 

Squadron  (air),  (2)  An  Infantry  Battalion  (ground),  and 
(3)  A  Rotary  Wing  Squadron  (air) .  This  was  done  by  design 
to  provide  as  comprehensive  a  cross  section  of  interviews  as 
possible  with  a  limit  of  three  case  studies. 

Those  individuals  interviewed  were  asked  four  general 
questions  that  provided  areas  of  focus  for  discussion. 

Every  evaluator  from  each  of  the  three  evaluation  teams  was 
interviewed  separately  such  that  his  response  was  autonomous 
and  not  influenced  by  those  of  his  peers.  Those  questions 
were : 

- What  are  the  elements  that  make  an  effective 

evaluator? 

- How  can  selection  of  potential  evaluators  best  be 

accomplished? 

- What  can  the  Evaluation/Exercise  Controller  or  Direc 

tor  do  to  make  selected  evaluators  more  effective? 


- Are  there  any  general  techniques  developed  and  used 

by  evaluators  that  make  for  more  effective  recording 


of  evaluation  results? 

1 .  Case:  Marine  Corps  Squadron  (FIXED  WING) 

A  MCCRES  was  conducted  on  an  A- 4  jet  aircraft  squad¬ 
ron  in  August  1983.  It  was  provided  for  and  regulated  by 
the  MCCRES  Order,  local  directives,  and  a  Letter  of  Instruc¬ 
tion  (LOI)  published  specifically  for  the  MCCRES  evaluation. 

The  management  control  structure  established  for  the 
execution  of  the  MCCRES  evaluation  was  identical  to  that  of 
Figure  3.2:  the  Commanding  General,  Fleet  Marine  Force,  was 
the  Evaluation/Exercise  Commander;  Commanding  General,  Wing 
was  the  Evaluation/Exercise  Director.  The  Tactical  Exercise 
Controller  and  Senior  Evaluator  were,  however,  one  in  the 
same.  The  evaluators  met  the  requirements  of  appropriate 
rank,  "recent  experience,"  and  "successful"  tour  in  the 
required  Military  Occupational  Speciality  (MOS) .  In  fact, 
the  professional  characteristics  of  the  evaluators  were 
identical  to  that  of  the  key  individuals  of  the  unit  being 
evaluated. 

There  was  no  school  held  for  MCCRES  evaluators,  only 
a  fifteen  minute  evaluator  "inbrief."  This  was  felt  justi¬ 
fied  by  the  Senior  Evaluator  (who  was  also  the  Tactical 
Exercise  Controller) ,  since  the  evaluators  being  used  were 
either  Weapons  Training  Instructor  (WTI)  qualified  or  had 
been  MCCRES  evaluators  in  the  recent  past. 


The  management  control  process  was  accommodated 
through  the  selection  of  the  MPS's,  their  observation,  grad 
ing,  and  the  passing  of  this  information  by  the  evaluators 
to  the  Senior  Evaluator.  A  list  of  MCCRES  Mission  Perfor¬ 
mance  Standards  (MPS's)  used  in  the  A-4  Squadron  MCCRES 
evaluation  is  presented  below: 

a.  Briefing/Debriefing 

b.  Aerial  Refueling 

c.  Coordinated  Strike 

d.  Rescap 

e.  Sauadron  Disaster  Plan 

f.  Aircrew  Knowledge  Exams 

g.  Aircraft  Surge  Capabilities 

h.  Close  Air  Support 

i.  Deep  Air  Support 

j.  Nuclear  Weapons  Delivery 

k.  Advanced  Weapons 

l.  Defensive  Tactics 

m.  Armed  Helo  Escort 

The  evaluation  feedback  process  was  provided  for  in 
the  published  LOI  and  included  those  levels  of  management 
critical  to  the  management  control  structure;  (1)  Senior 
Evaluator,  (2)  Tactical  Exercise  Controller,  (3)  Exercise 
Director,  and  (4)  Exercise  Commander.  These  levels  of  man¬ 
agement  are  depicted  in  Figures  3.1,  "Control  and  Feedback, 
and  3.2,  "MCCRES  Management  Control  Structure  Model,"  and 
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discussed  in  Chapter  III.  Also,  the  LOI  met  the  five  and 
ten  day  report  requirements,  as  stipulated  in  the  MCCRES 
order  and  discussed  in  Chapter  II.  This  provided  the 
required  feedback  information  as  established  in  the  manage¬ 
ment  control  process.  Figure  3.3,  "MCCRES  Management  Control 
Process  Model." 

During  the  conduct  of  the  MCCRES,  the  selection  of 
participants,  either  individual  or  aircrew,  in  the  squadron 
was  not  random  nor  did  100  percent  of  the  squadron  partici¬ 
pate.  Only  those  individuals  and  aircrews  declared  combat 
ready  through  another  reporting  system  (UNITREP)  were 
evaluated . 

The  essential  data  of  the  interview  information 
gathered  from  the  individual  MCCRES  evaluators,  based  on  the 
four  previous  questions,  is  presented  below. 

(1)  All  evaluators  expressed  strong  opinions  that 
both  overall  experience  and  a  recent  "successful"  tour 
flying  the  same  type  aircraft  as  the  evaluated  Squadron  ; 

was  paramount.  It  was  stated  that  if  these  requirements 
were  met  for  evaluators,  the  execution  of  more  effective 
grading  of  "technical"  tasks  was  possible. 

Overall  experience  was  important  in  that  it  pro¬ 
vided  credibility  to  the  evaluator.  Because  the  tactics  I 

of  air  war  have  been  undergoing  such  radical  change  in 
recent  years,  all  evaluators  felt  strongly  about  the 
need  for  potential  evaluators  to  have  served  in  a  flying  I 
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billet  very  recently  and  applied  these  new  tactics  in 
the  same  aircraft  as  found  in  the  unit  receiving  the 
MCCRES . 

(2)  There  was  a  general  conclusion  by  all  evaluators 
that,  because  of  the  level  of  experience  desired,  per¬ 
haps  only  Majors  and  Lieutenant  Colonels  should  be  used 
as  MCCRES  evaluators. 

(3)  3ecause  the  "community"  of  pilots  for  a  given 
type  of  aircraft  is  so  small  in  the  Marine  Corps,  the 
selection  of  potential  evaluators  is  critical  in  order 
to  negate  the  halo  effect,  central  tendency  effect, 
leniency  effect,  and  maintain  sufficient  autonomy.  The 
evaluators  felt  that  selection  should  be  made  from  units 
that  are  at  least  twice  removed  from  the  unit  receiving 
the  MCCRES.  This  would  require  one  Air  Wing  to  request 
assistance  from  another  Wing  in  providing  evaluators. 

All  evaluators  felt  this  would  greatly  enhance  the 
autonomy  of  evaluators  and  their  objectivity  in  grading 
a  given  MPS. 

(4)  It  was  their  stated  opinion  that  many  pilots 
meet  requirements  that  have  been  established  by  the 
MCCRES  Order,  but  lack  the  ability  to  objectively  grade 
tasks.  This  can  be  caused  by  many  evaluator  biases 
[Wheeler,  1983]  and  is  discussed  further  in  the  analysis 
portion  of  this  chapter. 
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(5)  All  individuals  interviewed  stated  that  ideally 
a  permanent  MCCRES  Staff  from  Headquarters  Marine  Corps 
would  be  most  effective  in  conducting  all  MCCRES  evalua¬ 
tions  throughout  the  Corps.  However,  realizing  the  man¬ 
power  constraints,  they  stipulated  a  reasonable  alterna¬ 
tive  would  be  a  permanent  MCCRES  staff  "core"  located 

at  each  major  command.  This  would  provide  better  conti¬ 
nuity  in  the  required  judgements  used  in  the  assignment 
of  a  grade  to  a  given  MPS  as  well  as  the  overall  unit 
grade.  The  interviewed  evaluators  stated  this  "core" 
could  then  be  augmented  by  individuals  from  units  that 
were  twice  removed  from  the  unit  receiving  the  MCCRES. 

(6)  Senior  Evaluators  and  Tactical  Exercise  Control¬ 
lers  that  were  interviewed  stated  that  the  authority  for 
selection  of  high  quality  evaluators  must  be  authorized 
at  the  Exercise/Evaluation  Director  '.evel,  with  total 
support  in  this  effort  actively  demonstrated  by  the 
Exercise/Evaluation  Commander. 

(7)  Evaluators  pointed  out  that  there  are  techniques 
used  during  the  actual  observation  of  MPS  tasks  and  the 
required  concurrent  marking  of  the  grade  of  these  tasks 
that  can  be  counter  productive.  They  stated  that  in 
many  instances  the  evaluator  can  mark  and  observe  con¬ 
currently  because  the  tempo  of  operations  is  such  that 
it  allows  for  this  technique  to  be  used.  However,  in  an 
operation  that  has  a  faster  tempo,  (i.e.,  coordinated 
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strike,  close  air  support,  nuclear  weapons  delivery,  and 
defensive  tactics,  of  listed  MPS's)  simultaneous  obser¬ 
vation  and  recording  is  impossible.  In  these  instances 
the  evaluator  must  observe  an  event  and  upon  conclusion 
quickly  mark  an  assigned  grade  on  his  checklist.  Each 
evaluator  agreed  that  effectiveness  is  compromised  to 
some  degree  by  not  being  able  to  observe  and  mark  con¬ 
currently.  While  the  evaluators  are  recording  this 
grade  some  new  event  is  ongoing  and  the  evaluators  can¬ 
not  give  it  their  full  attention.  To  compensate  in 
minimizing  their  effectiveness  in  this  regard  they  try 
to  develop  their  own  way  of  determining  how  much,  in  a 
given  observation  of  an  MPS  task,  they  can  realistically 
retain  before  it  must  be  recorded  on  the  checklist.  It 
was  pointed  out  by  the  evaluators  that  an  effective 
technique  such  as  color  coding  sections  of  work  sheets 
by  grouping  requirements  (so  each  group  stands  out 
because  of  its  color)  for  a  given  task  as  a  single 
observation  for  marking  may  be  something  that  could  be 
incorporated  into  inbrief  sessions  or  a  pre-MCCRES 
school . 

(8)  All  evaluators  pointed  out  there  is  no  pre-MCCRES 
testing  of  potential  evaluators  as  to  their  knowledge  in 
a  particular  Military  Occupational  Speciality  (MOS) . 
However,  they  stated  there  is  no  need  because  their  par¬ 
ticular  "community"  is  small  enough  that  each  pilot's 
knowledge  is  known  through  his  reputation  as  a  pilot. 


(9)  Evaluators  stated  they  would  like  to  be  briefed 
on  trends  of  past  MCCRES  discrepancies  found  in  similar 
units  and  also  any  possible  evaluator  bias  in  similar 
conditions.  This  would  give  them  a  "heads  up"  in  a 
given  area  and  they  could  be  aware  of  those  possible 
discrepancies  or  biases  during  the  MCCRES. 

(10)  There  is  a  split  in  stated  opinion  by  evalua¬ 
tors  as  to  which  technique  is  more  effective  in  deter¬ 
mining  a  "true"  grade  for  a  given  MPS  task.  In  the 
first  technique  the  evaluator  provides  immediate  feed¬ 
back  to  the  unit  on  observed  discrepancies  and,  if  not 
corrected  during  subsequent  observations  of  the  same  MPS 
task,  he  marks  the  task  as  a  fail  grade.  The  second 
technique  is  to  not  provide  the  initial  warning  and 
simply  grade  the  event  as  observed. 

2 .  Case:  Marine  Corps  Infantry  Battalion 

A  MCCRES  was  conducted  on  an  infantry  battalion  in 
September  1983.  It  was  provided  for  and  regulated  by  the 
MCCRES  Order,  local  directives,  and  a  Letter  of  Instruction 
(LOI)  published  specifically  for  the  MCCRES  evaluation. 

The  management  control  structure  established  for  the 
execution  of  the  MCCRES  evaluation  was  identical  to  that  of 
Figure  3.2:  the  Commanding  General,  Fleet  Marine  Force,  was 
the  Evaluation/Exercise  Commander;  Commanding  General,  Divi¬ 
sion,  however,  passed  the  authority  and  responsibility  of 
the  Evaluation/Exercise  Director  to  the  Regimental  Commander 
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The  Tactical  Exercise  Controller  and  Senior  Evaluator  was 
the  Regimental  Executive  Officer.  Ninety  percent  of  selected 
evaluators  were  of  appropriate  rank  and  experience.  They 
were,  however,  only  once  removed  (that  is,  within  the  same 
Regiment)  from  the  Battalion  receiving  the  MCCRES.  Two 
evaluators  of  eight  interviewed  had  an  MOS  that  was  not 
similar  tc  those  found  in  the  unit  receiving  the  MCCRES. 

There  was  no  school  held  for  evaluators  and  no 
explanation  was  given  as  to  why  there  was  none.  Six  of  the 
eight  evaluators  interviewed  had  no  previous  experience  as 
MCCRES  evaluators. 

The  management  control  process  was  provided  through 
the  selection  of  the  MPS's,  their  observation,  grading,  and 
the  transfer  of  this  information  by  the  evaluators  to  the 
Senior  Evaluator.  A  list  of  the  MCCRES  Mission  Performance 
Standards  (MPS)  used  in  the  MCCRES  evaluation  is  presented 
below . 

a.  Continuing  Actions  by  Marines 

b.  Command  and  Control 

c.  Fire  Support  Coordination 

d.  Heliborne  Assault 

e.  Movement  to  Contact 

f.  Attack 

g.  Defense 

h.  Retrograde  Operations 

i.  Mechanized  Operations 


j .  Amphibious  Raid 


The  process  tc  provide  feedback  of  evaluation  infor¬ 
mation  was  outlined  in  the  LOI  and  involved  those  levels  of 
the  management  control  structure,  as  established  in  Figure 
3.2,  "MCCRES  Management  Control  Structure  Model,"  which 
lists  those  individuals  that  influence  the  assignment  of  an 
assessment  to  a  given  MRS. 

The  main  issues  of  the  information  gathered  in  indi¬ 
vidual  interviews  with  evaluators  are  listed  below. 

(1)  All  evaluators  stated  they  felt  very  strongly 
that  overall  experience  and  a  recent  "successful"  tour 
in  the  infantry  was  critical.  However,  when  asked  to 
weigh  the  two,  overall  experience  was  listed  as  mere 
important . 

(2)  It  was  generally  felt  that  evaluators  functioned 
more  effectively  when  they  were  the  same  rank  as  that  of 
the  person  being  evaluated.  Therefore,  Captains  should 
be  used  to  evaluate  company  commanders.  Majors  tc  evalu¬ 
ate  functions  of  battalion  operations,  Lieutenant 
Colonel  to  be  used  as  a  senior  evaluator  and  counterpart 
to  the  battalion  commander. 

(3)  As  a  group,  these  evaluators  stated  they  were 
particularly  sensitive  to  autonomy  and  its  need.  Every 
evaluator  stated  there  was  a  significant  degradation  of 
evaluator  performance  if  the  evaluators  were  not  from 
units  twice  removed.  It  was  not  accomplished  in  this 
particular  MCCRES  and  all  evaluators  felt  a  kinship  to 
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those  individuals  being  evaluated  and  they  said  this 
ultimately  was  reflected  in  their  marking  of  MPS's. 

(4)  Evaluators  did  not  feel  the  need  for  a  permanent 
MCCRES  evaluation  staff  at  some  higher  headquarters. 

They  all  stated  there  was  a  significant  potential  learn¬ 
ing  experience  that  could  benefit  the  evaluator.  This 
experience  potentially  could  assist  the  current  evalua¬ 
tors  in  future  MCCRES  evaluations  in  which  "they"  would 
be  the  individuals  receiving  the  evaluation.  The 
experience  gained  as  evaluators  would  allow  them  to  be 
more  effective  in  preparing  for  "their"  MCCRES. 

(5)  Evaluators  stated  that  command  emphasis  and 

interest  in  the  MCCRES  evaluation  and  the  selection  of 
quality  evaluators  was  critical.  The  evaluators  per¬ 
ceived  only  general  interest:  by  higher  headquarters  for 
this  particular  MCCRES.  As  stated  by  one  evaluator, 
"everybody  hates  getting  tagged  to  be  a  MCCRES  evalua¬ 
tor . there  is  no  prestige  associated  with  the  job." 

(6)  Evaluators  stared  that  a  school  conducted  on 
evaluation  techniques,  scenario  of  the  MCCRES,  and  the 
MPS's  used,  would  be  helpful  toward  increasing  their 
effectiveness.  They  felt  however,  there  would  be  no 
need  to  teach  tactics,  given  the  potential  evaluators 
were  "quality"  selectees. 

One  teaching  technique  discussed  was  that  of 
using  a  given  Command  Post  Exercise  (CPX)  with  a 


developed  scenario  to  let  potential  evaluators  practice 
grading  a  given  MPS.  The  CPX  would  be  under  the  strict 
supervision  of  the  TEC  ana  Senior  Evaluator. 

(7)  All  evaluators  indicated  they  employed  the  tech¬ 
nique  of  providing  immediate  feedback  of  observed  dis¬ 
crepancies.  If  the  discrepancies  were  not  corrected 
during  subsequent  observations  of  the  same  MPS  task/’ 
requirement  the  evaluator  then  marked  the  task/require¬ 
ment  as  a  failure.  They  stated  there  are  two  ways  of 
looking  at  a  MCCRES  evaluation;  (1)  a  reporting  process, 
or  (2)  a  learning  experience.  Each  evaluator  interviewed 
stated  he  felt  it  was  mere  beneficial  to  use  the  "learn¬ 
ing  experience"  philosophy  and  icnore  the  "report  croc- 
ess,"  as  initially  described  in  the  first  case. 

3 .  Case:  Marine  Corps  Squadron  (ROTARY  WING) 

A  MCCRES  was  conducted  on  an  helicopter  squadron  in 
September  1983.  It  was  provided  for  and  regulated  by  the 
MCCRES  Order,  local  directives,  and  the  Letter  of  Instruc¬ 
tion  (LOI)  published  specifically  for  the  MCCRES  evaluation. 

The  management  control  structure  established  for  the 
execution  of  the  MCCRES  was  identical  to  that  of  Figure  3.2: 
the  Commanding  General,  Fleet  Marine  Force,  was  the  Evalua¬ 
tion/Exercise  Commander;  Commanding  General,  Wing  was  the 
Evaluation/Exercise  Director.  The  TEC  and  Senior  Evaluator 
were  the  same  individual.  Evaluators  were  of  the  appropri¬ 
ate  rank  and  experience  as  reauired  by  the  MCCRES  Order. 


There  was  no  school  held  for  evaluators,  only  a  shor 
"inbrief"  to  hand  out  MPS  requirements,  assign  evaluators  an 
discuss  the  conduct  of  the  MCCRES  scenario.  All  evaluators 
had  previous  experience  as  MCCRES  evaluators,  one  as  many  as 
fifteen  times. 

The  management  control  process  was  provided  through 
the  selection  of  the  MPS's,  their  observation,  grading,  and 
the  reporting  of  this  data  by  the  evaluators  to  the  Senior 
Evaluator.  A  list  of  MCCRES  Mission  Performance  Standards 
(MPS)  used  in  the  MCCRES  evaluation  is  presented  below. 

All  Helicopters 

a.  Continuing  Actions 

b.  Command  and  Control 

c.  Aircrew  Knowledge  Exam 

d.  Disaster  Plan 

e.  Electronic  Warfare 

Light  Helicopter 

a.  Heliborne  Assault 

b.  Command  and  Control 

c.  Visual  Reconnaisance  Operations 

d.  Combat  Resupply 

e.  Medical  Evacuation 

f.  Liaison  Carrier  Operations 

g.  Reconnaisance  Patrol  and  Reaction  Force 
Operations 

h.  Night  Operations 


Attack  Helicocter 


a.  Ileliborne  Assault 

b.  Assault  Support  Helo  Support 

c.  Convoy  Escort 

d.  Medical  Evacuation  Escort 

e.  Reconnaisance  Patrol  and  Reaction  Force 
Operations 

f.  Close  In  Fire  Support  and  Airborne  Fire  Support 

g.  Visual  Reconnaisance  Operations 

h.  Night  Operations 

The  process  established  to  provide  feedback  of 
evaluation  data  was  outlined  in  the  LOI  and  involved  these 
levels  of  management  control,  as  established  in  Figure  3.2, 
"MCCRES  Management  Control  Structure  Model,"  that  influenced 
the  assignment  of  an  assessment  to  a  given  MRS. 

The  main  points  gathered  through  interviews  of  indi¬ 
vidual  evaluators  are  listed  below. 

(1)  Experience.  The  views  of  the  interviewed 
evaluators  were  essentially  the  same  as  discussed  in  the 
other  two  cases.  The  summarized  opinion  of  the  evalua¬ 
tors  was  that  experience  is  a  function  of  rank  and  is 
the  most  critical  element  an  evaluator  must  have  to  be 
effective . 

(2)  Rank.  As  in  the  previous  two  cases,  the  evalua¬ 
tors  felt  rank  to  be  the  second  most  important  criteria 
because  of  its  close  relationship  to  experience. 
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(3)  Autonomy,  as  it  relates  to  evaluators  and  their 
effectiveness,  was  clearly  stated  by  evaluators  as  being 
third  in  importance.  This  element's  contribution  to 
evaluator  effectiveness  is  fully  discussed  in  the  first 
case . 

(4)  Interviewed  evaluators  did  not  express  any  opin¬ 
ion  concerning  the  benefits  of  either  a  permanent  or 
augmented  MCCRES  evaluation  staff. 

(5)  Interviewed  evaluators  felt  strongly  about  the 
need  for  command  interest  and  support  to  be  actively 
demonstrated  to  all  concerned  within  the  MCCRES  evalua¬ 
tion  process. 

(6)  A  restatement  of  comment  (7)  in  the  first  case 
on  evaluator  developed  techniques  for  marking  fast  tempo 
operations,  i.e.,  grouping  and  color  coding  given  task/ 
requirements  on  work  sheets  such  that  the  marking  of  a 
grade,  resulting  from  an  observation,  is  logically  broke: 
down  for  mere  efficient  marking  and  resulting  evaluator 
effectiveness . 

(7)  There  was  no  evaluator  school  given  before  this 
MCCRES  was  conducted.  Evaluators  felt  this  would  be 
beneficial  if  techniques  for  evaluator  effectiveness 
were  stressed  in  a  class  presentation  and  then  those 
effectiveness  principles  were  tested. 

(8)  The  evaluators  perceived  being  effective  through 
two  distinct  approaches.  In  the  first  approach  the 


evaluator  simply  "reports"  on  his  observations  as  they 
occur.  In  the  second  approach,  the  entire  process  of 
preparation  and  then  the  conduct  of  the  MCCRES  by  the 
evaluated  unit  is  looked  on  as  "a  learning  experience." 
This  approach  is  facilitated  through  evaluators  provid¬ 
ing  informal  feedback  of  mission  effectiveness  to  the 
unit  as  the  MCCRES  scenario  progresses.  In  this  latter 
approach,  if  proper  corrective  action  is  not  taken  after 
evaluator  feedback  is  initially  given,  the  unit  is 
adversely  graded  for  that  MPS  event. 

(9)  Interviewed  evaluators  stated  that  much  of  the 
overall  MCCRES  effectiveness  depends  on  the  TEC  and  his 
ability  to  properly  develop  a  scenario  that  is  both 
challenging  and  realistic  to  a  particular  unit's  mission 
and/or  location,  and  yet  not  repetitive  relative  to  the 
units  last  MCCRES.  The  basic  theme  is  that:  any  unit  can 
be  graded  as  "combat  ready"  if  the  scenario  used  for 
evaluation  is  the  same  "canned  package"  the  unit  has 
seen  in  past  MCCRES  evaluations.  When  this  happens,  the 
interviewed  evaluators  said  that  they  have  a  tendency  to 
lose  objectivity  and  mark  a  given  MPS  as  it  has  been 
marked  in  the  past  with  the  same  unit,  thus,  severely 
degrading  evaluator  effectiveness. 

C.  ANALYSIS  OF  CASES 

The  purpose  of  the  analysis  is  to  determine  if  comments 

made  by  interviewed  MCCRES  evaluators  are  valid  for 


incorporation  into  either  the  structure  or  process  of  the 
management  control  system  used  in  the  selection  and  use  of 
MCCRES  evaluators.  If  incorporation  is  accomplished,  it 
must  be  done  in  a  way  so  that  it  will  enhance  the  overall 
effectiveness  of  the  current  MCCRES  system  through  proper 
application  of  the  principles  of  management  control  theory. 

For  the  purpose  of  this  analysis  only  a  comment  that  was 
initiated  and  made  by  at  least  one  member  from  each  separate 
MCCRES  evaluation  team  is  considered  in  the  analysis.  In 
other  words,  a  valid  comment  must  have  been  initiated  and 
discussed  on  at  least  three  separate  occasions  by  one  or 
more  members  from  each  of  the  three  MCCRES  evaluator  teams 
that  were  interviewed. 

Comments  were  then  weighted  and  are  listed  below  in 
order  of  importance,  with  _1  indicating  "very  important"  and 
_8  indicating  "important."  The  weighting  is  based  on  the 
evaluation  of:  (1)  how  many  times  the  issue  was  brought  up, 
(2)  how  strongly  an  individual  voiced  his  opinion  on  the 
subject,  and  (3)  who  brought  up  the  issue  (i.e.,  an  experi¬ 
enced  evaluator's  strong  support  of  an  issue  was  weighted 
more  heavily,  relative  to  his  experience,  than  a  less 
experienced  evaluator  who  also  felt  strongly  about  some 
other  issue) .  Evaluator  comments  to  be  considered  are 
listed  below. 

Command  Interest  and  Support  (1) .  There  must  be  an 
active  demonstration  of  total  support  by  both  the 
Evaluation  Commander  and  the  Evaluation  Director. 


to  be  a  function  of  one  another,  they  enhance  evaluator 
effectiveness  or  at  least  apparently  provide  credibility 
to  those  being  evaluated. 


Evaluator  Autonomy"  (3)  .  Evaluators  must  have  autonomy 
to  objectively  observe  MCCRES  MPS  events  and  effectively 
mark  them. 

Evaluator  School  (4) .  A  school  provided  before  the 
MCCRES  is  critical.  Subjects  such  as  evaluator  effec¬ 
tiveness,  evaluator  perceptions,  MCCRES  scenario  to  be 
used,  review  of  MPS 1 s  to  be  used,  techniques  to  mark 
fast  tempo  operations,  and  opportunity  to  practice 
observations  and  do  marking  should  be  considered. 

Valid  Scenario  (5>.  An  effective  TEC  must  develop  a 
realistic  and  challenging  MCCRES  scenario.  This 
scenario  must  be  based  on  the  unit's  mission,  location, 
and  its  probable  deployable  geographical  area. 
Determination  by  Exercise  Director  as  to  Purpose,  or 
Tone  of  Evaluation  (6) .  The  Exercise  Director  must 
decide,  publish,  and  then  enforce  whether  the  evaluation 
is  being  conducted  as  an  opportunity  to  report  on  the 
unit  or  an  opportunity  to  provide  a  learning  experience 
for  the  unit  receiving  the  MCCRES. 

Past  Evaluation  Trends  (7) .  Either  during  the  evaluator 
school  or  the  inbrief,  evaluators  should  be  given  a  list 
of  reoccurring  MPS  discrepancies  that  similar  units  have 


made  during  recent  MCCRES  evaluations.  This  list  would 
allow  the  evaluators  to  focus  on  those  discrepancies, 
knowing  there  is  a  high  probability  of  unit  readiness 
problems  in  these  identified  areas. 

External  Evaluator  to  MCCRES  (3) .  An  evaluator  to 
evaluate  the  evaluators,  provided  from  Headquarters, 

Fleet  Marine  Force  could  potentially  provide  impetus  at 
the  levels  of  planning,  selection,  schooling,  and  conduct 
of  a  MCCRES,  for  a  more  effective  overall  MCCRES  system. 
This  evaluator  would  report  his  findings  directly  to  the 
Exercise/Evaluaticn  Commander. 

1 .  Management  Control:  Structure  Enhancement  of  MCCRES 
As  discussed  in  Chapter  III,  management  control 
structure  identifies  the  authority  and  responsibility  of 
management  to  accomplish  tasks  that  contribute  to  achieve¬ 
ment  cf  organizational  goals.  Command  Interest  and  Support 
(1) ,  Evaluator  Rank/Experience  (2) ,  and  External  Evaluator 
to  MCCRES  (8) ,  discussed  in  the  previous  section,  contribute 
directly  to  the  enhancement  of  the  MCCRES  structure  as  it 
relates  to  management  control. 

The  first,  and  perhaps  the  most  important.  Command 
Interest  and  Support  (1) ,  is  simply  a  restatement  or  reempha¬ 
sis  of  the  existing  requirement  to  correctly  execute  those 
responsibilities  established  through  the  use  of  assigned 
authority  as  discussed  and  depicted  in  Figure  3.2,  "MCCRES 
Management  Control  Structure  Model." 
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The  second,  Evaluator  Rank/Experience  (2),  is  criti¬ 
cal  in  providing  the  objectivity  required  for  meeting  the 
goal  of  achieving  a  "true"  marking  for  a  given  MRS  observa¬ 
tion.  If  the  appropriate  maturity  of  an  evaluator  (based  to 
some  degree  on  his  rank)  and  sufficient  technical  knowledge 
(based  on  his  experience)  is  present,  the  achievement  of 
this  goal  is  more  closely  realized. 

The  last  contribution  to  structure.  External  Evalua¬ 
tor  to  MCCRES  (8) ,  identifies  the  need  to  establish  a  modi¬ 
fication  to  the  existing  structure  depicted  in  Figure  3.2 
such  that  an  autonomous  evaluation  of  the  overall  MCCRES 
process  itself  is  possible,  regardless  of  the  unit  being 
evaluated.  It  is  this  new  element  that  can  be  added  to  the 
structure  .that  will  provide  for  a  more  objective  and  compre¬ 
hensive  feedback  of  information  concerning  the  effectiveness 
of  the  Exercise/Evaluation  Director,  TEC,  and  evaluators  in 
their  contribution  no  the  overall  evaluation  system.  This 
information  would  be  reported  directly  to  the  Exercise/ 
Evaluation  Commander  by  the  External  Evaluator. 

The  effect  of  these  three  factors  on  the  structure 
model  itself  will  be  discussed  in  Chapter  V;  "A  Model,"  when 
the  model,  as  depicted  in  Figure  3.2,  is  reviewed  for 
reorganization  to  accommodate  this  analysis. 

2 .  Management  Control:  Process  Enhancement  of  MCCRES 
As  discussed  in  Chapter  III,  the  management  control 
process  provides  for  those  actions,  or  the  transferring  of 


information  and  its  use,  that  take  place  to  accomplish  those 
goals  set  by  management.  Command  Interest  and  Support  (1) , 
Evaluator  Rank/Experience  (2) ,  Evaluator  Autonomy  (3) , 
Evaluator  School  (4),  Valid  Scenario  (5) ,  Determination  by 
Exercise  Director  as  to  Purpose,  or  Tone  of  Evaluation  (6) , 
and  Past  Evaluation  Trends  (7) ,  all  factors  discussed  previ¬ 
ously,  contribute  directly  to  the  enhancement  of  the  MCCRES 
process  as  it  relates  to  management  control  as  depicted  in 
Figure  3.3,  "MCCRES  Management  Control  Process  Model." 

The  first,  and  perhaps  most  important.  Command 
Interest  and  Support  (1),  contributes  to  the  structure  of 
MCCRES  but  it  also  significantly  influences  or  emphasizes 
the  exchange  of  information.  If  command  interest  and  sup¬ 
port  is  emphasized,  the  qualify  of  data  collected  and 
reported  will  be  considerably  upgraded.  For  this  reason  it 
is  included  in  the  process  review  of  MCCRES. 

The  second,  Evaluator  Rank/Experience  (2),  influences 
the  effectiveness  of  an  evaluator  in  the  performance  of  his 
duty  as  an  observer  and  marker  of  what  a  unit  accomplished. 
Experience,  as  a  function  of  rank,  helps  to  create  the 
credibility  needed  by  the  evaluator  so  that  he  spends  less 
time  justifying  his  marking  and  more  time  concentrating  on 
the  actual,  accurate  observation  of  MPS  events  as  they 
occur.  This  process,  in  aggregate,  contributes  to  a  higher 
quality  of  data  gathered  and  a  more  efficient  system. 
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The  third,  Evaluator  Autonomy  (3),  directly  influ¬ 
ences  the  performance  of  an  evaluator  more  than  any  other 
single  factor  so  state  interviewed  evaluators.  3ecause  of 
this,  the  evaluators  strongly  recommend  that  there  is  a  need 
for  the  "twice  removed"  rule.  Evaluators  state  that  if  this 
rule  is  applied  it  allows  them  to  be  as  objective  in  their 
marking  of  a  given  MPS  as  is  possible.  If  this  is  not  done, 
the  issue  of  the  "halo  effect,"  as  discussed  by  Wheeler 
(1983)  becomes  paramount  and  there  is  a  significant  degra¬ 
dation  of  objectivity  in  marking  an  MPS. 

The  fourth,  Evaluator  School  (4) ,  serves  to  make  the 
potential  evaluator  aware  of  evaluator  bias,  more  efficient 
and  effective  marking  techniques,  and  the  opportunity  to 
practice  evaluator  techniques  and  evaluation  principles. 

This  school  could  provide  a  means  for  the  collected  evalua¬ 
tion  information  that  is  exchanged  between  different  levels 
of  management  to  be  viewed  as  creditable  data  for  use  by 
those  who  ultimately  assign  the  overall  MCCRES  grade. 

The  fifth,  Valid  Scenario  (5) ,  provides  for  a  mere 
realistic,  creditable  scenario  and  because  of  this  motivates 
all  involved  in  the  MCCRES  to  support  the  overall  evaluation 
process  and  to  contribute  a  greater  degree  of  support  to  the 
accomplishment  of  its  purpose:  the  measurement  of  a  unit's 
readiness . 

The  sixth,  Determination  by  the  Exercise  Director 
as  to  Purpose  of  MCCRES  (6) ,  determines  how  the  evaluators 
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will  approach  the  marking  of  the  tested  unit  in  its  perform¬ 
ance  of  accomplishing  a  given  MPS.  It  is  the  "tone"  of  how 
the  Exercise  Director  feels  a  MCCRES  should  be  viewed. 

Whether  his  view  is  that  of  MCCRES  as  a  "report"  or  a 
"learning  experience,"  he  must  effectively  convey  this  view 
to  his  TEC,  Senior  Evaluator,  and  Evaluators.  The  process 
for  each  view  is  significantly  different  and  provides  a  very 
different  type  of  end  product. 

The  seventh.  Past  Evaluation  Trends  (7) ,  is  an  oppor¬ 
tunity  for  the  evaluators  to  concentrate  on  certain  informa¬ 
tion,  and  if  done  correctly  provide  a  high  probability  of 
greater  marginal  return  on  available  evaluator  time  allo¬ 
cated  for  a  given  MCCRES. 

The  influence  of  these  seven  factors  on  the  process 
model  itself  will  be  discussed  in  Chapter  V;  "A  Model,"  when 
the  model,  as  depicted  in  Figure  3.3,  is  reviewed  for 
reorganization  as  a  result  of  this  analysis. 

D .  SUMMARY 

The  interviews  with  individuals  from  each  MCCRES  evalua¬ 
tion  team  from  three  different  major  commands  resulted  in  a 
set  of  issues  and  improvements.  The  issues  and  improvements 
that,  in  the  evaluator's  opinion,  could  increase  the  effec¬ 
tiveness  of  MCCRES,  but  yet  be  common  to  all  Marine  Corps 
commands,  was  discussed  in  detail.  These  eight  factors  were 
then  weighted  and  ranked,  based  on  gathered  information  from 
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interviewed  evaluators  and  applied  criteria  established 
measuring  evaluator's  perception  of  importance.  The  factors 
were  then  related  to  one  of  the  two  elements  of  management, 
control;  structure  or  process. 

It  was  determined  in  the  analysis  that  two  of  the  eight 
factors  contributed  to  both  the  structure  and  process  models 
of  management  control.  To  summarize  which  model  each  item 
of  discussion  is  related  to.  Figure  4.1  is  provided. 


EVALUATOR  COMMENTS  AND  ISSUES 
(FACTORS) 


STRUCTURE  ENHANCEMENT 


PROCESS  ENHANCEMENT 


MORE  IMPORTANT 


1*  Command  interest  &  Support  1.  Command  interest  & 

Support 

2.  Evaluator  Rank/Experience  2.  Evaluator 

Rank /Experience 

3 .  Evaluator  Autonomy 

4.  Evaluator  School 

5.  Valid  Scenario 

6.  Purpose/Tone  of 
Eval 

7 .  Past  Evaluation 
Trends 

3.  External  Evaluator  to  MCCRES 


IMPORTANT 


Figure  4.1  Importance  and  Category  of  Issues  for 
Enhancement  of  the  MCCRES  System 


V.  A  MODEL 


A.  GENERAL 

The  question  posed  in  this  chapter  is  one  of:  "Can  a 
better  mouse  trap  be  built?"  The  more  specific  question  is 
one  asked  in  Chapter  I  of  this  thesis:  "Can  a  model  be 
developed  to  assist  in  the  efficient  selection  and  education 
of  potential  MCCRES  evaluators  so  that  the  evaluators  will 
perform  at  a  predetermined  level  of  effectiveness?" 

However,  before  pursuing  this  question,  there  is  a  need 
to  respond  to  an  implied  question  found  within  the  question 
posed  for  this  thesis.  The  implication  is  that  of,  "is  tne 
current  evaluation  system  any  good?"  In  response  to  this, 
all  respondents  to  field  interviews  indicated  that  currently 
evaluators  are  effective  and  the  system  used  to  select  and 
train  them  is  efficient.  However,  all  indicated  it  could 
be  improved  upon.  Therefore,  a  better  mouse  trap  is  the 
issue  of  this  chapter. 

The  Marine  Corps  Order  that  established  and  regulates 
the  MCCRES  system,  (MCO  3501.2)  stipulates  a  desired  level 
of  performance  effectiveness  for  its  evaluators  through  the 
use  of  Evaluator  Performance  Standards  (EPS) .  The  Order 
points  out  that  for  a  successful  MCCRES,  three  types  of 
evaluators  are  required:  (1)  'umpire,  (2)  performance 

evaluator,  and  (3)  exercise  controller.  These  three  types 
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com- 


model,  or  models,  that  have  the  potential  to  assist  the 
mander  in  his  selection  and  use  of  MCCRES  evaluators.  the 
development  of  the  model  is  acccmolished  throuah  the  use  of 
the  elements  of  management  control:  structure  and  process. 

B.  MODEL  DEVELOPMENT 

In  cue  discussion  of  the  development  of  the  model  to 
enhance  the  commander's  ability  to  more  effectively  select 
and  use  potential  MCCRES  evaluators  there  are  two  areas  of 
focus.  First,  is  the  MCCRES  organizational  structure  as  in 
relates  to  authority  and  responsibility .  Model  development 
is  accomplished  by  using  knowledge  gained  from  previous  dis¬ 
cussions:  (1)  management  control  theory  and  its  application 

to  structure ,  as  presented  in  Chapter  III;  (2)  analysis  of 
cases,  presented  in  Chapter  IV;  and  (3)  "the  MCCRES  manage¬ 
ment  control  structure  model"  (Fig.  3.2)  ,  which  ,'epicts  from 
where  and  to  when  the  evaluation  data  flows  in  the  MCCRES 
organization . 

As  a  result  of  the  analysis  in  the  previous  chapter, 
key  individuals  in  the  existing  organizational  structure  are 

(1)  The  MCCRES  Evaluaticn/Sxercise  Director  and  his 
influence  on  the  selection  of  potential  MCCRES 
evaluators . 

(2)  The  Tactical  Exercise  Controller  in  his  role  of 
influencing  the  level  of  control  and  education  of  the 
selected  evaluators. 
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(3)  The  external  evaluator  to  the  MCCRES  organiza¬ 
tion,  who  measures  the  overall  effectiveness  of  the 

MCCRES  system  itself  and  then  formally  reports  this  to 

the  Exercise/Evaluation  Commander. 

The  second  area  of  focus  is  the  MCCRES  process  as  it 
relates  to  the  effective  collection  and  use  of  evaluation 
data.  This  model  development  is  accomplished  through  knowl¬ 
edge  gained  from  previous  discussions:  (1)  management  con¬ 

trol  theory  and  its  application  to  process ,  as  presented  in 
Chapter  III;  (2)  the  analysis  of  cases,  presented  in  Chapter 
IV;  and  (3)  "the  MCCRES  management  control  process  model 
(Fig.  3.3),  which  depicts  what  MCCRES  does  with  the  evalua¬ 
tion  information.  The  resulting  product  of  the  analysis  is 
the  key  contributing  factors  to  this  process.  These  factors 
are : 

(1)  The  need  for  effective  evaluator  schooling. 

(2)  Using  top  qualiry  people  as  evaluators. 

(3)  Providing  for  better  MCCRES  scenarios  and  techniques 
for  marking  it. 

(4)  Superiors  who  potentially  can  influence  a  given 
MCCRES  by  actively  demonstrating  concern  and 
interest . 

It  is  the  combination  of  both  elements;  key  individuals 
(structure)  and  key  contributing  factors  (process),  that 
bring  focus  on:  (1)  the  required  actions  of  certain  indi¬ 
viduals  (billet/oositions)  ,  (2)  where  those  individuals  fit 


into  the  organizational  structure,  (3)  the  providing  for  an 
effective  school  and  its  teachings,  and  (4)  a  scenario  that 
will  provide  for  a  more  realistic  opportunity  to  measure 
readiness,  that  result  in  identifying  the  required  ingredi¬ 
ents  for  constructing  "a  better  mouse  trap." 

C.  MODEL  DESIGN 

The  purpose  of  the  proposed  models  is  to  allow  the  com¬ 
mander  to  effectively  select  potential  evaluators,  and  then 
provide  a  means  to  enhance  their  effectiveness  through  edu¬ 
cation.  This  purpose  can  be  accomplished  more  effectively 
through  the  modification  of  two  previously  presented  MCCRES 
models;  (1)  management  control  structure  for  MCCRES,  (Fig. 

3.2) ;  and  (2)  management  control  process  for  MCCRES,  (Fig. 

3.3)  . 

1 .  Management  Control  Structure  Model  for  MCCRES 

The  model  that  is  used  to  reflect  a  mere  effective 
method  for  the  use  of  authority  and  responsibility  in  the 
conduct  of  a  given  MCCRES  is  proposed  in  Figure  5.2.  This 
model  is  a  modification  of  Figure  3.2;  justification  for 
the  changes  is  based  on  conclusions  from  the  analysis  dis¬ 
cussed  in  Chapter  IV. 
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Figure  d.2  Modified  MCCRSS  Management  Control  Structure 
Model 

The  model  depicted  in  Figure  5.2  reflects  the  adap¬ 
tation  of  Figure  3.2  to  include  the  organizational  require¬ 
ment  for  the  restructuring  of  MCCP.ES  as  required  by  factor 
(8)  "External  Evaluator  uo  MCCRES”  of  the  analysis  as  pre¬ 
sented  in  Chapter  IV.  The  modified  model.  Figure  5.2,  with 
its  addition  of  the  "external  evaluator,"  meets  the  required 
need  for  reorganization  such  that  the  organizational  struc¬ 
ture  can  provide  the  support  needed  if  an  evaluation  of 
MCCRES  is  to  be  accomplished  as  a  complete  and  separate  sys¬ 
tem  to  be  used  for  effective  measurement  of  a  unit's  readi¬ 
ness.  This  new  requirement,  of  an  external  evaluator  to 
evaluate  the  effectiveness  of  the  MCCRES  Director,  his  TEC 
and  its  evaluators,  provides  a  means  for  needed  feedback 
information.  This  feedback  is  reauired  for  controls  to 


enhance  the  effectiveness  of  the  overall  system.  These  con¬ 
trols  provide  for:  (1)  effective  use  of  evaluators  who  have 
appropriate  rank  and  experience,  and  (2)  renewed  interest 
by  all  concerned,  through  information  feedback  and  the  sub¬ 
sequent  applied  control  by  the  MCCRES  Exercise/Evaluation 
Commander.  Those  decisions  made  as  the  result  of  this  addi¬ 
tional  control  feature  will  result  in  higher  quality  MCCRES 
scenarios  and  more  effective  overall  unit  evaluation. 

2 .  Management  Control  Process  Model  for  MCCRES 

The  model  that  is  used  for  the  discussion  of  illus¬ 
trating  a  more  effective  use  of  MCCRES  evaluation  informa¬ 
tion  is  depicted  in  Figure  5.3.  This  model  is  identical  to 
that  of  Figure  3.3,  however,  it  is  presented  again  so  that 
emphasis  and  discussion  can  be  focused  on  critical  points 
within  the  model.  The  points  in  Figure  5.3  not  germane  no 
the  results  of  the  analysis  presented  in  Chapter  IV,  and 
therefore  not  discussed,  are  those  areas  with  diagonal  lines 
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Figure  5.3  MCCRES  Management  Control  Process  Model  (revised) 
adapted  from  Ramanathan  (1982,  p.  176) 


The  model  presented  in  Figure  5.3  is  useful  in  serv¬ 
ing  as  a  tool  to  focus  the  discussion  on  the  enhancement  of 
the  MCCRES  process.  The  model  provides  a  means  to  emphasize 
corrections  to  the  MCCRES  "process''  enhancement,  as  stipu¬ 
lated  in  the  analysis  of  Chapter  IV. 

As  stated  in  the  analysis  from  the  previous  chapter. 
Chapter  IV,  factors ;  Command  Interest  &  Support  (1) , 


Evaluator  Rank/Experience  (2) ,  Evaluator  Autonomy  (3) , 
Evaluator  Schooling  (4),  Valid  Scenario  (5),  Purpose/Tone 
of  Evaluation  (6) ,  and  Past  Evaluation  Trends  (7) ,  can  con¬ 
tribute  to  the  improved  effectiveness  of  the  MCCRES  process 
which  contributes  ultimately  to  the  successful  operation  of 
the  MCCRES.  For  the  model  to  be  valid,  it  must  accommodate 
each  of  the  listed  factors  one  through  seven. 

Factors  1  and  2,  "Command  Interest  and  Support,"  and 
"Evaluator  Rank/Experience,"  respectively,  cannot  be  tied  to 
any  singular  part  of  the  process  model.  However,  the 
quality  of  both  affect  the  entire  MCCRES  process  and  the 
ultimate  effectiveness  toward  its  output,  measuring  unit 
readiness.  By  emphasizing  command  interest  and  the  quality 
of  people  with  commensurate  experience  throughout  the  MCCRES 
process,  the  expected  results  would  be  some  level  of  MCCRES 
performance  above  what  it  was  previously. 

Factor  3,  "Evaluator  Autonomy,"  affects  the  areas  of 
"MCCRES  Performance"  and  "MCCRES  Evaluating  and  Reporting. " 
It  is  in  these  two  areas  where  autonomy  and  its  direct  con¬ 
tribution  to  objective  grading  may  provide  the  greatest 
benefit.  It  is  the  autonomy  an  evaluator  achieves  through 
the  "twice  removed"  rule  that  allows  the  reduction  of  peer 
pressure  to  a  point  that  objective  marking  is  better  served 
and  ultimate  overall  system  enhancement  achieved. 


Factor  4,  "Evaluator  School,"  will  affect  the  proc¬ 
ess  by  contributing  to  the  facilitation  of  improved  evaluator 


effectiveness.  If  "Evaluator  Perception"  of  his  duties  and 
his  abilities  to  effectively  observe  and  accurate  grade 
given  events  are  enlightened  through  the  teaching  of  evalua¬ 
tion  principles  and  techniques,  subsequent  increased  effec¬ 
tiveness  in  evaluator  performance  will  result.  If  the 
evaluator  is  also  taught  the  principles  of  bias  that  effect 
"Evaluator  Preference"  and  "Evaluator  Acceptance, "  both  on 
the  pare  of  the  evaluator  and  the  evaluated  unit,  the  result 
will  be  a  more  "accurate"  or  effective  system  for  evaluation, 
i.e.,  enhanced  "MCCRES  Performance.” 

Factor  5,  "Valid  Scenario,"  effects  the  model  only 
after  strategic  planning  takes  place.  Certainly  it  is 
critical  during  the  "MCCRES  Programming"  phase,  given  that 
some  portion  of  the  overall  MCCRES  program  is  specifically 
tailored  each  time  a  given  unit  is  evaluated.  The  results 
of  a  well  thought  cut  scenario  for  a  given  unit  is  the  prod¬ 
uct  of  a  valid  "Specific  Program  Objective"  for  measurement 
of  combat  readiness  that  is  designed  for  that  unit.  The 
evaluation  report  that  goes  to  Headquarters,  Marine  Corps 
also  provides  the  means  for  feedback  cn  the  quality  of  this 
effort  to  upgrade  and  maintain  a  valid  evaluation  program. 

Factor  6,  "Determination  by  the  Exercise  Director 
as  to  Purpose  of  MCCRES,"  affects  the  evaluation  report. 

The  report  is  a  valid  statement  of  what  was  actually  accom¬ 
plished  during  the  MCCRES  process.  However  the  process  may 
be  viewed  in  at  least  two  ways: 

S  '• 


(1)  is  the  MCCRES  to  be 


an  observation  of  a  unit's  readiness  performance  and  a 
report  on  that  observation,  or  (2)  is  it  to  be  an  operation 
in  which  there  is  a  free  flowing  exchange  of  information  and 
ideas  between  the  unit  and  the  MCCRES  team  with  the  end  prod¬ 
uct  for  the  evaluated  unit  culminating  in  the  overall 
improvement  in  its  proficiency  at  performing  its  combat  mis¬ 
sion.  Either  question  serves  the  Order  on  MCCRES,  as  it  is 
written,  but  this  "tone"  needs  to  be  stipulated  and  enforced 
as  the  standard  for  evaluators  to  use  during  the  conduct  of 
a  MCCRES. 

Factor  7,  "Past  Evaluation  Trends,"  is  an  issue  that 
falls  well  within  the  model's  "MCCRES  Performance"  block. 
Analysis  of  past  trends  can  provide  for  more  efficient  use 
of  the  evaluator's  time  and  thus  provide  a  more  comprehen¬ 
sive  evaluation  of  MPS  tasks  the  evaluator  must  observe  and 
mark . 

D.  SUMMARY 

In  conclusion,  this  chapter  has  evaluated  those  comments 
and  issues  collected  from  evaluator  interviews.  With  the 
validation  of  the  eight  factors  (as  determined  in  the  analy¬ 
sis)  as  the  corner  stone  for  improving  the  effective  use  of 
evaluators  new  models  were  developed.  Each  factor  has  been 
incorporated  into  discussions  concerning  the  illustration 
of  two  newly  developed  models.  The  models  represent  both 
the  structure  and  process  for  improved  management  control 
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of  the  MCCRES  and  its  resulting  contribution  to  effective 
measurement  and  its  reporting  of  unit  readiness. 

It  is  the  conclusion  of  the  investigation  of  this  thesis 
that  the  two  models,  Figures  5.2  and  5.3,  represent  a  means 
to  increase  the  effectiveness  of  the  MCCRES  system.  However 
this  conclusion  and  recommendations  made  as  the  result  of 
the  analysis  and  subsequent  developed  models,  as  reflected 
in  Figures  5.2  and  5.3,  are  the  subject  of  Chapter  VI;  "Con¬ 
clusion  and  Recommendation." 


VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A .  GENERAL 

The  objective  of  this  thesis  was  to  provide  an  answer  to 
the  question:  "Can  a  model  be  developed  to  assist  in  the 
efficient  selection  and  education  of  potential  MCCRES  evalua 
tors  so  that  the  evaluators  will  perform  at  a  predetermined 
level  of  effectiveness?" 

If  a  model  can  be  developed,  a  secondary  question  must 
then  be  asked:  "What  attributes  of  the  management  control 
system  (i.e.,  its  structure  and  process)  will  this  modal 
provide?"  The  attributes  can  provide  the  concerned  com¬ 
mander  with  a  technique  to  use  for  more  effective  management 
of  resources. 

The  first  part  of  this  chapter  summarizes  the:  (1)  find 

ings  of  the  investigation,  (2)  results  of  rhe  analysis,  and 
(3)  subsequent  development  of  a  new  model  to  accommodate  the 
results  of  the  analysis.  Based  on  this  summary  the  chapter 
makes  recommendations  that  provide  the  commander  a  tool  to 
be  used  in  the  selection  of  a  more  effective  MCCRES  evalua¬ 
tor.  Recommendations  for  additional  research  are  also 
provided . 


B.  CONCLUSIONS 


It  was  found  that  there  were  eight  factors  that  could 
enhance  both  the  ef fectiveness  of  a  commander  in  his  selec¬ 
tion  of  evaluators  and  their  subsequent  performance.  The 
eight:  factors ,  are:  (1)  Command  Interest  and  Support, 

(2)  Evaluator  Rank/Experience,  (3)  Evaluator  Autonomy, 

(4)  Evaluator  Schooling,  (5)  Valid  Scenario,  (6)  Purpose/ 
Tone  of  Evaluation,  (7)  Past  Evaluation  Trends,  and  (3) 
External  Evaluator  to  MCCRES.  The  eight  factors  were 
weighted  based  upon  three  criteria.  The  three  weighting 
criteria  are:  (1)  frequency  that  the  factor  was  initiated 

and  discussed  by  the  interviewed  evaluator,  (2)  how 
strongly  the  evaluator  voiced  his  opinion  about  the  f actor , 
and  (-3)  the  degree  of  significance  and  perceived  validity 
of  the  factor  by  the  evaluators.  These  criteria  provided 
a  means  to  evaluate  the  significance  and  validity  of  com¬ 
ments  in  terms  of  the  level  of  evaluator  experience. 

Based  upon  an  analysis  of  the  factor 1 s  validity,  as 
supported  by  management  control  theory  and  the  determina¬ 
tion  of  the  appropriate  weight,  each  f actor  was  assigned  a 
numerical  value  of  one  (most  important)  through  eight 
(important).  Each  factor  was  then  placed  in  descending 
order  of  importance.  This  ordering  facilitated  the  com¬ 
parison  of  the  relative  affect  each  factor  had  on  the  cur¬ 
rently  used  MCCRES  system  models  (Fig.  3.2  and  Fig.  3.3). 
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The  models  currently  used  reflect  an  effective  manage¬ 
ment  control  system  for  MCCRES.  The  model  presented  in 
Figure  3.2  illustrates  the  existing  structure  of  the  system 
and  the  model  presented  zn  Figure  3.3  illustrates  the 
existing  process.  Even  though  the  current  MCCRES  is  effec¬ 
tive,  the  means  by  which  the  evaluators  are  selected  and 
trained  can  be  improved.  The  analysis  in  this  thesis  indi¬ 
cated  adjustments  to  the  current  process  and  structure  will 
result  in  a  better  use  of  the  evaluator  as  a  resource.  The 
new  structure  (Fig.  5.2)  and  process  (Fig.  5.3)  models  were 
developed  to  incorporate  the  requirement  of  the  eight  fac¬ 
tors  identified  by  the  interviewed  evaluators.  The  new 
models  provide  for  system  improvements  by  accommodating  the 
requirements  of  the  eight  factor s  identified  by  the  thesis 
research.  Furtru  ,  the  models  provide  the  commander  an 
opportunity  to  incrementally  adjust  the  system  to  meet 
unique  situations  or  requirements.  By  judiciously  applying 
one  factor ,  several  factors ,  or  all  factors  and  their  attri¬ 
butes,  the  commander  can  exercise  a  varying  decree  of  change 
to  the  current  MCCRES  system.  The  extent  of  conversion  a 
commander  desires  depends  on  the  availability  of  additional 
resources  and  the  commander's  commitment  to  support  the 
eight  factors  as  a  means  to  enhance  MCCRES  evaluator  selec¬ 
tion  and  his  subsequent  operational  effectiveness. 
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direct  contribution  to  the  MCCRES  effort.  This  investiga¬ 
tion  should  be  continued,  witn  similar  efforts  directed  at 
the  Umpire  and  Exercise  Controller  and  their  contribution  to 
an  effective  evaluation  system.  Upon  conclusion  of  these 
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efforts  there  could  be  a  consolidation  of  the  results  of  each 
of  the  three  studies  to  determine  if  there  are  common  bound¬ 
aries  and  principles  equally  applicable  to  each  type  of 
evaluator  depicted  in  Figure  5.1,  The  Complete  Evaluator. 

(b)  It  is  recommended  there  be  a  continuation  of  the 
investigation  initiated  with  this  thesis.  Further  study 
could  provide  a  different  perspective  on  the  potential 
effectiveness  of  the  models  developed  in  this  thesis.  The 
use  of  research  methods  other  than  the  case  study  method 
might  be  used  to  achieve  another  perspective  of  the  effec¬ 
tiveness  of  MCCRES  and  to  validate  the  models  developed  in 
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